-- Logs begin at Thu 2022-09-01 08:21:01 UTC, end at Thu 2022-09-01 09:39:44 UTC. -- Sep 01 08:51:06.509179 dv-cinder-820027-23-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 08:51:06.518999 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 08:51:06 2022] *** Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: nodename: dv-cinder-820027-23-iscsi Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: machine: x86_64 Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: clock source: unix Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: pcre jit disabled Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: detected number of CPU cores: 4 Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: current working directory: / Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: detected binary path: /usr/bin/uwsgi-core Sep 01 08:51:06.521123 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: your processes number limit is 47680 Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: your memory page size is 4096 bytes Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: detected max file descriptor number: 2048 Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: lock engine: pthread robust mutexes Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: thunder lock: enabled Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 08:51:06.521779 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 08:51:06.560169 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: Python main interpreter initialized at 0x557e5cd4e9c0 Sep 01 08:51:06.560169 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: python threads support enabled Sep 01 08:51:06.560169 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: your server socket listen backlog is limited to 100 connections Sep 01 08:51:06.560169 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: your mercy for graceful operations on workers is 90 seconds Sep 01 08:51:06.560506 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 08:51:06.560624 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: *** Operational MODE: preforking *** Sep 01 08:51:06.560881 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: *** uWSGI is running in multiple interpreter mode *** Sep 01 08:51:06.560881 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: spawned uWSGI master process (pid: 81622) Sep 01 08:51:06.561069 dv-cinder-820027-23-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 08:51:06.562161 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: spawned uWSGI worker 1 (pid: 81633, cores: 1) Sep 01 08:51:06.562161 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: spawned uWSGI worker 2 (pid: 81634, cores: 1) Sep 01 08:51:06.562161 dv-cinder-820027-23-iscsi devstack@keystone.service[81622]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 08:51:10.877656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 08:51:10.877656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: from cryptography.utils import int_from_bytes Sep 01 08:51:10.878473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 08:51:10.878473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: from cryptography.utils import int_from_bytes Sep 01 08:51:10.880042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 08:51:10.880042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: from cryptography.utils import int_from_bytes Sep 01 08:51:10.881137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 08:51:10.881137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: from cryptography.utils import int_from_bytes Sep 01 08:51:10.928324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 08:51:10.928514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 08:51:10.928707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] command line args: [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 08:51:10.928974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 08:51:10.929152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ================================================================================ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 08:51:10.929407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] admin_token = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.929653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_dir = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.929840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.930021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_source = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.930200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] debug = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.930398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.930705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 08:51:10.930705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 08:51:10.930917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] command line args: [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 08:51:10.931150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 08:51:10.931351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ================================================================================ {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 08:51:10.931562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] admin_token = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] config_dir = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.931964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_config_append = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] config_source = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_dir = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] debug = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.932997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.933968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_config_append = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_dir = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_file = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.934936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.935933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] publish_errors = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.936963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] standard_threads = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.937823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] publish_errors = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_journal = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_json = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.938986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_stderr = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_syslog = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.939793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.939921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] standard_threads = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.940045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.940167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.940283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.940398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.940518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.940646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.940770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.940887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] use_journal = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.940887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] use_json = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.941101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.941216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] use_stderr = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.941411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.941537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] use_syslog = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.941660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.941789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 08:51:10.941912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.942028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.942158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.942276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.942397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.943965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.944958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.945904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.946977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.947922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.948976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.949905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.950912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.951969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.952994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.953945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.954993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.955978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.956914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.957970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.958921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.959993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.960940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.961921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.962981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.963954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.964997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.965909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.external = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.966962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.password = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.967212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.967314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.token = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.external = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.972956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.password = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] auth.token = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.973888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.974958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.975977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.976985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.977958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.978933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.979986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.980971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.981959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.982944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.983965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.984935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.985890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.986872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.987965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.988989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.989142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.989222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.989915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.990873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.991901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.992990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.993941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.994816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.995837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.996975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.997965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.998928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.999066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.999202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.999344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:10.999452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.003978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] role.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] role.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.004878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.005969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.006945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.007985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.008926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.009964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.010939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.011970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.012993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.connection = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.013926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.014868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.015019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.015128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.015207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 08:51:11.015306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.015588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.015833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.016068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.016313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.016549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.016798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.017037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.017281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.017512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.017751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.017984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 08:51:11.018154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 08:51:11.044206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.044430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.044656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.053625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.054633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.054875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.055197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.063962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.081883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.082115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.082811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.083044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.136396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.140744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.194799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.195039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.195687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.195926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.373326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.373555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.373759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.373876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.373990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.374841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.401539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.401661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.401798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.401883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.401972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.402843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.413806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.413987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.429550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.429684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.429840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.429937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81633) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.430936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 08:51:11.448198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.448403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 08:51:11.448619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.448767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.448856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 08:51:11.448932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 08:51:11.449211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.449424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.449508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.449628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 08:51:11.449714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.449900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.449991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.450160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.450417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.450672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.460106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.460383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.460654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.460935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.461021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.461234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.461506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.461787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.467863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.468769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.470714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.470993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.473695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.473695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.473900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.474005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.474957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.475281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.496924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.498040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.498379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.498660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.499520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.499802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.503146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.503434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.503717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.504469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.504754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.505668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.510734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.511016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.511970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.512261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.514938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.515250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.515535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.515814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.516086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.516456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.516748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.517031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.517315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.517614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.518613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.534033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.534313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.535947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.536409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.538131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.540546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.541201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.541500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.543719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.544016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.545732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.548276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.548862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.551642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.552498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.552802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.553169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.553456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.554231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.554231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.555564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.555871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.556160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.556441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.556733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.557009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.570315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.570608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.570876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.571163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.587726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.587978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.588364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.588860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.588860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.589144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.596751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.597036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.597321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.597600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.597910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.598185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.598483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.600804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.601084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.601362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.601642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.601921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.602198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.602500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.613748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.617614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.619530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.620482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.621918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.622403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.623406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.623690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.623993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.625365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.626918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.628428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.628716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.629005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.631487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.632989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.635873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.636092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.636265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.636445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.636725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.637684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.640624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.640922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.641207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.641483 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.644582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.644875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.645159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.649405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.649696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.649977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.652089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.652324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.652607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.652886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.653162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.653444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.653729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.654008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.654289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.654564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.654834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.655095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.655383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.655663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.657115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81633) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 08:51:11.657398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.657678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.657952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.658293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.658570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.658841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.659138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.659522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.659804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.660077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.660339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.660602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.660884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81633) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 08:51:11.690279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557e5cd4e9c0 pid: 81634 (default app) Sep 01 08:51:11.694576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eb0f-e03b-4847-b35f-03faca4ccf7f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:11.694868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eb0f-e03b-4847-b35f-03faca4ccf7f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:11.695160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eb0f-e03b-4847-b35f-03faca4ccf7f None None] PATH_INFO: `/v3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:11.696574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557e5cd4e9c0 pid: 81633 (default app) Sep 01 08:51:11.699329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu Sep 1 08:51:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:19.062343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7215-61b3-4192-954b-877304e67704 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:19.062848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7215-61b3-4192-954b-877304e67704 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:19.062935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7215-61b3-4192-954b-877304e67704 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:19.067340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:19.073642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:19.074062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:19.074182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:19.143877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: INFO dbcounter [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] Registered counter for database keystone Sep 01 08:51:19.163793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG oslo_db.sqlalchemy.engines [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81634) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 08:51:19.170135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writer thread running {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 08:51:19.348937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG passlib.handlers.bcrypt [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81634) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 08:51:19.350323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG passlib.handlers.bcrypt [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81634) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 08:51:19.409400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:19.410977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:19.538099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 08:51:20.019519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b7215-61b3-4192-954b-877304e67704 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:20.090929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: INFO dbcounter [None req-c12b7215-61b3-4192-954b-877304e67704 None None] Registered counter for database keystone Sep 01 08:51:20.112298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG oslo_db.sqlalchemy.engines [None req-c12b7215-61b3-4192-954b-877304e67704 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81633) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 08:51:20.118835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writer thread running {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 08:51:20.242290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b7215-61b3-4192-954b-877304e67704 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:20.247095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:20.247812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:20.248206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:20.248521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:20.258298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:20.310833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING keystone.common.rbac_enforcer.enforcer [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 08:51:20.311177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346267a1-9b58-45ba-98e6-51d12a94626e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:20.321567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Sep 1 08:51:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 08:51:20.327522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:20.342919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a7a11-bc7c-4bd2-9710-653c99305f53 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:20.347434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:20.348507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:20.348806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:20.349061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] PATH_INFO: `/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:20.351126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] RBAC: Authorizing `identity:get_project(project_id=0c919d0d1fdc41adb58cac59aa4513ea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:20.404970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 08:51:20.405318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721551e-4c92-4b54-a63d-e3bc0c989d8d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:20.409931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu Sep 1 08:51:20 2022] GET /identity/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:22.268352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f925e-ccd7-4a1b-99ad-65ec40ebf9cb None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:22.268870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f925e-ccd7-4a1b-99ad-65ec40ebf9cb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:22.268952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f925e-ccd7-4a1b-99ad-65ec40ebf9cb None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:22.273892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:22.279084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:22.279526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:22.279657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:22.300117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:22.301598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:22.309633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 08:51:22.573854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f925e-ccd7-4a1b-99ad-65ec40ebf9cb None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:22.586609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f925e-ccd7-4a1b-99ad-65ec40ebf9cb None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:22.590801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:22.591432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:22.591713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:22.591985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:22.599625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:22.601434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530397f4-06e7-4cc4-aeac-36992ba7bc82 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:22.610072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Sep 1 08:51:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 08:51:22.615448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:22.625775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c2cb3-c3c6-4491-a3ed-c594e87d90de None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:22.630065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:22.631204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:22.631484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:22.631757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:22.633060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:22.634890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b442d-dbb3-4824-ab9b-e3dec564463d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:22.636456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu Sep 1 08:51:22 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:24.597776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451cf0-9285-445f-ba79-5c87128388f8 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:24.598279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451cf0-9285-445f-ba79-5c87128388f8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:24.598360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451cf0-9285-445f-ba79-5c87128388f8 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:24.600206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:51:24.605315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:24.605746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:24.605995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:24.623507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b410-7b3b-4ba1-8b7e-7520518e7cad None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:24.623785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b410-7b3b-4ba1-8b7e-7520518e7cad None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:24.624058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b410-7b3b-4ba1-8b7e-7520518e7cad None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:24.625260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:24.625359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:51:24.626438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:24.631411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:24.631696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:24.631975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:24.674879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG passlib.handlers.bcrypt [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81633) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 08:51:24.675300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG passlib.handlers.bcrypt [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81633) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 08:51:24.706865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:24.724910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:24.726474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:24.824393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:24.971316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:24.995852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.074886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c32603-28db-4981-b6cf-3c1d98b837cf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.087239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.108091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.108719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.108998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.109270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.120970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:25.129948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:25.130875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:25.131815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:25.136954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.189004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd51dd-e4f5-4642-ae4b-aaa49b27f9fa None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.191259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.220548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.226597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.227498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.227784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.228054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.239135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613dca4-5e55-4569-b1be-0536cfc1ac37 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.273154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.273753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.274035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.274313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.274975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:25.275317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.276734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.288938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:25.294198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.296352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.298857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.397940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ce855-3de1-45e0-b597-c0cb892a5a43 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.399469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:51:25.404982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.407767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.427692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.428314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.428593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.428863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.430003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.431481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.433039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:25.438604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.453073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.502541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec90b-a137-47bf-92e6-9d95f3d40139 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.536529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.537472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.537623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb58098-98f4-43ba-949d-aff40c27616e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.537733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.538002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.557559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.560673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.571900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.572536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.572842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.573117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.574242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.575734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.577288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:51:25.580316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:25.581370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:51:25.582259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:51:25.582259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:51:25.582259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:25.582259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:25.584197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.587725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:25.642949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.666042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:25.684337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9cfe8-3471-474a-ba7c-6c732b8c5ee9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.688211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e0b02-62ce-4c7b-bc32-aae68bed1968 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:25.718553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.719184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.719465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.719735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.720457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.domain.id=default, user.id=78b783de521c4bc3ade218e12aef00c7) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:25.720766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.721707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:25.722247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ada4f4-3b1b-4f78-b88b-393fe52a74df None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.722341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:25.722567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:25.722833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:25.723496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:25.723795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:25.725455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837f12d-238b-4a0d-a8b7-be5ae5aec923 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:25.729220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/role_assignments?user.id=78b783de521c4bc3ade218e12aef00c7&scope.domain.id=default&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:25.745332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:28.005143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f729d-0a39-4400-b1ac-63e6d54a4d1a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.006706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f729d-0a39-4400-b1ac-63e6d54a4d1a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.007224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f729d-0a39-4400-b1ac-63e6d54a4d1a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.008785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:28.014307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7fba7-799a-47b2-b8a6-658861ede31f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.015198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7fba7-799a-47b2-b8a6-658861ede31f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.015668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7fba7-799a-47b2-b8a6-658861ede31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.035074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c9a7fba7-799a-47b2-b8a6-658861ede31f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:28.037212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c9a7fba7-799a-47b2-b8a6-658861ede31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:28.120599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:28.132572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a858645-c539-4c22-893c-90570e629e57 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.133144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a858645-c539-4c22-893c-90570e629e57 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.133612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a858645-c539-4c22-893c-90570e629e57 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.135165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:28.149447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf8d2-2ff6-4d7f-ad46-a3dbf6b2e9a4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.150032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf8d2-2ff6-4d7f-ad46-a3dbf6b2e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.150507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf8d2-2ff6-4d7f-ad46-a3dbf6b2e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.171733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-958cf8d2-2ff6-4d7f-ad46-a3dbf6b2e9a4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:28.173245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-958cf8d2-2ff6-4d7f-ad46-a3dbf6b2e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:28.233118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:28.268054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 12/25] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:28.285912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e4a2-0f21-4fba-9ed9-06bb7e6c41e4 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.286570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e4a2-0f21-4fba-9ed9-06bb7e6c41e4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.287052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63e4a2-0f21-4fba-9ed9-06bb7e6c41e4 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.288679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 14/26] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:28.293898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664fd43-cbc7-4951-9821-2d0c141aa52d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.294184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664fd43-cbc7-4951-9821-2d0c141aa52d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.294457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664fd43-cbc7-4951-9821-2d0c141aa52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.324874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-0664fd43-cbc7-4951-9821-2d0c141aa52d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:28.326109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0664fd43-cbc7-4951-9821-2d0c141aa52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:28.418992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 13/27] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:28.650146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcb38-389f-481f-8ea9-9e6385714632 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.650486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcb38-389f-481f-8ea9-9e6385714632 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.650684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fcb38-389f-481f-8ea9-9e6385714632 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.652416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 15/28] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:28.657199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.657805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.658806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fcb38-389f-481f-8ea9-9e6385714632 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:28.659730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.698717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:28.701045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:28.813058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 14/29] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:28.814421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:28.867461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:28.877043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fcb38-389f-481f-8ea9-9e6385714632 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:28.939410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:28.940042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:28.940322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:28.940594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:28.961508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:28.966882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:28.967763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:28.968617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 16/30] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:28.969265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:28.977812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c2e0-af99-4d3e-9857-8f2d777b6d3e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:28.978548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:29.017370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.018023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.018293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.018536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.019230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.020625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.106560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b8e8-8d4e-4eca-ad00-b660711bc222 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.142299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.142940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.143253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.143528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.156850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.158343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:51:29.174949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:29.175899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:51:29.176720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:51:29.176720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:51:29.176720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:29.176720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 17/31] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:29.176958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.237504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:29.284495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d57e4e-5ce0-44fa-b82a-1413cff7f4ae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.308418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 15/32] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 08:51:28 2022] POST /identity/v3/roles => generated 219 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:29.310525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.331237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.331874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.332156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.332433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.333132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:29.334042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.338464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.355785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 18/33] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:29.356785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.491442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:29.497118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1260e-0b1c-4ae9-9fd7-a64a06a9185d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.520214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4724a-b250-4fee-8cef-7a70166ae074 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.543076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:29.545951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.546573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.546853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.547149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.547817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.551035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da2765-9ee4-45d9-9d7f-d46ea4a59aa1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.565182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.565799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.566070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.566340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.566990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:29.567367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.568805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.587680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 19/34] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:29.588509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.632756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 16/35] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 08:51:29 2022] POST /identity/v3/roles => generated 225 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:29.634344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48ba9-bce4-49b3-91e7-4883fbd82895 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.634733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48ba9-bce4-49b3-91e7-4883fbd82895 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.635017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b48ba9-bce4-49b3-91e7-4883fbd82895 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.640016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 17/36] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:29.642180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b48ba9-bce4-49b3-91e7-4883fbd82895 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.713627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc178605-227e-449c-a5e6-b6a78ad89ac5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.762456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.763188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.763574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.763963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.765243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.766728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a80419-b014-4fe4-9e68-6963f1f4be75 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.768457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 20/37] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:29 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:29.769725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050942-7670-4712-8d79-add989453fa0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.770017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050942-7670-4712-8d79-add989453fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.770291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050942-7670-4712-8d79-add989453fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.779223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b48ba9-bce4-49b3-91e7-4883fbd82895 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.789182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-25050942-7670-4712-8d79-add989453fa0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:29.790571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-25050942-7670-4712-8d79-add989453fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:29.820273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:29.821130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.821421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.821704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.822614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:29.824318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0164-43ec-4cd4-90ee-4c09e0f65af6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:29.846318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:29.871842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 21/38] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:29.873067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e5b6-7bda-4cae-a281-00dd65b4bff9 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.873354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e5b6-7bda-4cae-a281-00dd65b4bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.873626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115e5b6-7bda-4cae-a281-00dd65b4bff9 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.875020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 22/39] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:29.875574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115e5b6-7bda-4cae-a281-00dd65b4bff9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:29.975223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 18/40] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:51:29 2022] POST /identity/v3/projects => generated 301 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:29.975223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:29.975223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:29.975223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:29.989713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115e5b6-7bda-4cae-a281-00dd65b4bff9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:29.998343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:30.000368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:30.002951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:30.033476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:30.034189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:30.034532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:30.034813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:30.036330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:30.038801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:30.041238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 23/41] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:30.047038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:30.102379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 19/42] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:30.113521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:30.153338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:30.182460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd084f-bd05-4d91-a7d4-38d0d1ed97f7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:30.222342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:30.223012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:30.223320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:30.223596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] PATH_INFO: `/v3/domains/default/users/78b783de521c4bc3ade218e12aef00c7/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:30.225807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=78b783de521c4bc3ade218e12aef00c7, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:30.227359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:30.269645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372da72-6a9c-4737-bf9f-d2ac1faafa50 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:30.302984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:30.303611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:30.303896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:30.304197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:30.304853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:30.306236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:30.377021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:30.383534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 20/43] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 08:51:30 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:51:30.384302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:30.455512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 24/44] 10.222.0.46 () {58 vars in 1361 bytes} [Thu Sep 1 08:51:30 2022] PUT /identity/v3/domains/default/users/78b783de521c4bc3ade218e12aef00c7/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:51:30.500562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c0c34-5c70-4d9d-88c9-f1fb5718aa6d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:30.541141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:30.541800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:30.542087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:30.543203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:30.543561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:30.545047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50050f-2b3b-4e0e-975e-473cca1d0e20 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:30.546918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 21/45] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:30.554383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:30.763785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7cc6-fbbd-4192-96ae-b0df00978361 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:30.766120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:30.797129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:30.797783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:30.798080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:30.798623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:30.799484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:30.801628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e371263-30fb-48cb-aa82-66d290943450 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:30.888693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 25/46] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:51:30 2022] POST /identity/v3/projects => generated 312 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:32.376277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cfd88-93e6-46ff-8823-b45b5a58aa02 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:32.376798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cfd88-93e6-46ff-8823-b45b5a58aa02 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:32.376798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cfd88-93e6-46ff-8823-b45b5a58aa02 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:32.378149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 22/47] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:32.384066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ac39-df7d-48e0-8d82-0761763f884d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:32.384446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ac39-df7d-48e0-8d82-0761763f884d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:32.384623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2ac39-df7d-48e0-8d82-0761763f884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:32.403317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2bf2ac39-df7d-48e0-8d82-0761763f884d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:32.404811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2bf2ac39-df7d-48e0-8d82-0761763f884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:32.470568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 26/48] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:32.738918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cfd88-93e6-46ff-8823-b45b5a58aa02 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:32.785453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8fb65-4013-417e-aa89-35ff1edab391 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:32.785850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8fb65-4013-417e-aa89-35ff1edab391 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:32.786194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8fb65-4013-417e-aa89-35ff1edab391 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:32.788383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 27/49] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:51:32.793355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:32.793642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:32.793924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:32.807374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:32.812670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:32.813861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:32.842277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cfd88-93e6-46ff-8823-b45b5a58aa02 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:32.876318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:32.876939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:32.877221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:32.877490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:32.881883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 28/50] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:32.888838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:32.894778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:32.895852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:32.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 23/51] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:32.901943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:32.919826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:32.999716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf92c31-dd97-47cb-9425-099e9c8db833 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.031915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.032572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.032854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.033130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.033794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:33.034099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.035529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.044454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 29/52] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:33.049937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:33.148294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e1ebb-1c6f-43b4-a0f6-8da12db0f97a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.162749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:33.174290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:33.183298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.183979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.184257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.184528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.185893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.187369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.188926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 24/53] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:33 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:33.194449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:33.243903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:33.264871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bb1e9-cdaf-4f9c-9db2-3400a3a2e001 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.295041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb59ee-24a9-41dc-8569-35f23f581632 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.299474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.300112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.300396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.300675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.301794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.303325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.304908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 30/54] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:33.310392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:33.330390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.331046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.331483 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.331758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.332909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.335005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.336825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 25/55] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:33.342305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:33.405901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:33.431272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd9c5a-53e3-4f40-9d83-b198e1238a75 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.471472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.472110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.472384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.472668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.473358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.475431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541e613-371a-4b5b-8a40-edea8d297ab2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.554741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101f723-7008-4a70-a993-42dd4d8c53a3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:33.580432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:33.590246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 31/56] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Sep 1 08:51:33 2022] POST /identity/v3/projects => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:51:33.593783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:33.594471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:33.594795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:33.595074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:33.595855 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.domain.id=default, user.id=78b783de521c4bc3ade218e12aef00c7) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:33.596205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:33.597890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f64c8-b4f7-47b6-8332-dd0dbe76e44d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:33.608674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 26/57] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Sep 1 08:51:33 2022] GET /identity/v3/role_assignments?user.id=78b783de521c4bc3ade218e12aef00c7&scope.domain.id=default&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 530 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:35.526268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057ef9-f301-4316-a39e-9025bee43f1a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:35.526809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057ef9-f301-4316-a39e-9025bee43f1a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:35.526809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca057ef9-f301-4316-a39e-9025bee43f1a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:35.528221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 32/58] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:35.533372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:35.533734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:35.533934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:35.552902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:35.554081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:35.619314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 27/59] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:35.897138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057ef9-f301-4316-a39e-9025bee43f1a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:35.997225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca057ef9-f301-4316-a39e-9025bee43f1a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:36.029622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896d964-de62-4930-b80d-4586de94146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:36.030239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d964-de62-4930-b80d-4586de94146e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:36.030523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d964-de62-4930-b80d-4586de94146e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:36.030800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d964-de62-4930-b80d-4586de94146e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:36.031946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896d964-de62-4930-b80d-4586de94146e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:36.033405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896d964-de62-4930-b80d-4586de94146e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:36.034970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 33/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:36.041129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:36.095184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:36.138014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c50a6b-e5bc-4098-b8e4-a0606e567005 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:36.170116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:36.170659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:36.170941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:36.171247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:36.171900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:36.173338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49063c8-810a-4089-91f1-5b60e0e36b99 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:36.281593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 28/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:51:36 2022] POST /identity/v3/users => generated 275 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:51:39.298068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e453-6574-47b2-836b-f1d64374dd7a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.298957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e453-6574-47b2-836b-f1d64374dd7a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.299422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e453-6574-47b2-836b-f1d64374dd7a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.300926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 34/62] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:39.307074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c8ab-e1b5-4ac6-8d3b-7bd97d41a9ab None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.307531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c8ab-e1b5-4ac6-8d3b-7bd97d41a9ab None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.307700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56c8ab-e1b5-4ac6-8d3b-7bd97d41a9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.335803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-4b56c8ab-e1b5-4ac6-8d3b-7bd97d41a9ab None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:39.336992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4b56c8ab-e1b5-4ac6-8d3b-7bd97d41a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:39.373458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d441-d34f-4c04-bc2c-79768464d5ac None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.373968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d441-d34f-4c04-bc2c-79768464d5ac None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.374420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9d441-d34f-4c04-bc2c-79768464d5ac None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.376235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 35/63] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:39.383133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.383627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.384168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.416843 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:39.418073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:39.442279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 29/64] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:39.511426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:39.518821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 36/65] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:39.801662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:39.801844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb3955-a384-4875-88e2-60ba424939bd None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.802161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb3955-a384-4875-88e2-60ba424939bd None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.802416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb3955-a384-4875-88e2-60ba424939bd None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.803894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 30/66] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:39.815561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a17b-f292-4e7f-991f-91857c7687ca None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.816192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a17b-f292-4e7f-991f-91857c7687ca None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.816471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898a17b-f292-4e7f-991f-91857c7687ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.842906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3898a17b-f292-4e7f-991f-91857c7687ca None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:39.845837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3898a17b-f292-4e7f-991f-91857c7687ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:39.888318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:39.919211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 31/67] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:39.920466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840fe2-2c69-43a3-9927-0542c286b100 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.920756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840fe2-2c69-43a3-9927-0542c286b100 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.921225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840fe2-2c69-43a3-9927-0542c286b100 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 32/68] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de220c4-bdb0-41ae-b6e3-838b1eada19e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de220c4-bdb0-41ae-b6e3-838b1eada19e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de220c4-bdb0-41ae-b6e3-838b1eada19e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 33/69] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:39.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de220c4-bdb0-41ae-b6e3-838b1eada19e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:39.935546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b294483-c64d-417c-9737-403367ba0b3e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:39.973827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:39.975319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.975531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.975807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:39.986833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ff8fb6c9-2655-4f89-8a55-3f61cf3b510f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:39.992443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:39.993295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:39.994060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:39.996357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 37/70] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:39.998022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd073cd-cc94-4781-a8c8-a05c36f57a44 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:39.998467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd073cd-cc94-4781-a8c8-a05c36f57a44 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:39.998879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd073cd-cc94-4781-a8c8-a05c36f57a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.018222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5cd073cd-cc94-4781-a8c8-a05c36f57a44 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:40.019423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5cd073cd-cc94-4781-a8c8-a05c36f57a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:40.043520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de220c4-bdb0-41ae-b6e3-838b1eada19e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.071023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.078988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.079643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.081852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.081852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.088147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 38/71] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:40.089387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.089672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.089970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.096676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:40.102648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:40.103557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:40.104333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 34/72] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:40.104977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.117297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:40.118791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:40.136458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.214008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 39/73] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:40.214712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.221409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c56725-bf2e-4e09-8c47-47bcd250ebd8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.263924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.264859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.265343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.265846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.266713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:40.267386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.268947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.282988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 35/74] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.288439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.340804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67f34-2ea9-430f-b513-09d402caacbb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.362688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.382943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.383594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.383877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.384154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.384852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:40.385165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.386571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.396174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 40/75] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.397273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.399820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.407323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd04652-7b27-4561-b26c-90bb4d350782 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.449162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.449872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.450142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.450419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.460845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:40.465849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:40.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:40.467558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 36/76] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:40.469031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.507405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d27a7-825d-45c6-9467-7675cf7c7692 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.541328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.541955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.542235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.542508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.543691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.545141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.546857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 41/77] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.547566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.573337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d04dc-bdce-4c1e-ad46-12a3249348a5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.594338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.607562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.608188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.608470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.608744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.619840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:40.624779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:40.625614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:40.626373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 37/78] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:40.627190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.642403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.648722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b057df9-734e-497b-ab62-a3ce977ecf57 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.683145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.683765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.684088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.684368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.697226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.698673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.700315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 42/79] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.704429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.728840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d854100-6f65-4d78-93f2-3043ee84d4f8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.762629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.764157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.764157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.764157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.764449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:40.764751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.766136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 38/80] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.775502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.808200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3682c-1788-4c10-a2ca-62f3e46c660a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.810449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.841257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.841882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.842165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.842442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.843603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.845107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.846626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 43/81] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.847468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:40.855347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:40.878210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf345c-a265-4ccf-964d-99f846fdc26b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.912410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.913014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.913295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.913578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.946454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a9dc2-7512-4462-a1da-965a0d3812d0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:40.980452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.981768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:40.981940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.982372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:40.982657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:40.982934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:40.983625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:40.983934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:40.985321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:40.998755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 44/82] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:40.998755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.028153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.057655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.080614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 39/83] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:41.081493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.097801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e837bf3-6020-4982-8a52-333a3d75fb3a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.131275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.131886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.132170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.132449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.179274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd98a82-b236-454e-ada8-a56231b3f925 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.200419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.201894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.212578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.213203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.213486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.213761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.214880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.216360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.217952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 40/84] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:41.218685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.234694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.267385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.299721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 45/85] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:41.299721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.322338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40f00-2b61-43f5-b43b-a8ccbca07082 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.356620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.357156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.357443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.357719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.358370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.359863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.397727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fde2a-6760-4cef-b212-3ad2ba334d0b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.431599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.432215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.432500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.432778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.433904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.435438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.437017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 46/86] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:41.437769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.440204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.482133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 41/87] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:51:41 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:41.483032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.561583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7377-2058-47bb-9a3a-3599b0b6a688 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.597934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.601770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6897-b108-4251-86f4-722840335c2e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.602777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.603472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.603985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.604300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.606441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.612742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.612742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 47/88] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:41.612742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.651563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.652403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.652689 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.652961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.692479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.731975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8900-6e1d-491b-bfd4-4ae0f2f37e59 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.736142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.737842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.780554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.781025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.781302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.781582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.782729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:41.784269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:41.785941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 48/89] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:41.786941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.864216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.868943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 42/90] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:41.869614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:41.913346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4707f3b-bb4a-4823-804a-e7d54322f607 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:41.960891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:41.962245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:41.962718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:41.963207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:41.975125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:41.988960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20548f-5246-4216-9949-df1fb7bd7c4c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.028442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.029055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.029340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.029615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.030517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=78b783de521c4bc3ade218e12aef00c7) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:42.030826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.032879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.045000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 43/91] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/role_assignments?user.id=78b783de521c4bc3ade218e12aef00c7&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:42.045000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:42.085831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.087649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.175237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:42.240420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 49/92] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:42.241223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:42.245239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c5c57-38ac-46c6-9d9d-f5d07fb61d53 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.284542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.285160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.285444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.285791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.287137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:42.288425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.293819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:42.295709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.308744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 44/93] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:42 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:42.311382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:42.429524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6dad2-c5d4-4881-a59d-270a2c9da9a7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.507933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.508583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.508866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.509144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] PATH_INFO: `/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.512913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] RBAC: Authorizing `identity:get_project(project_id=f6cd27fa85264ac58298f2d2b0667bc7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.517335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.527287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 50/94] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:42 2022] GET /identity/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:42.531329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc6db-920b-4ccf-a37d-4e8df2f4413f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.537237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:42.585929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:42.595451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.596141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.597715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.597715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.597949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.600348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.602138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 45/95] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:42 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:42.608649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:42.726013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7ec45-ef4a-4634-91f8-a950128054c0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.762989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.764153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.764578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.765019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.766228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:42.766681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.768969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229d8d7-37d1-4251-8cf6-d2fa8f58bbf5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.775722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 46/96] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:42 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:42.785269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0176b-fa13-48d7-ac21-d45e152a48ee None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:42.795185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:42.834148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:42.835047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:42.835492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:42.835937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:42.836811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=f6cd27fa85264ac58298f2d2b0667bc7, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:42.837265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:42.838932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e544a-3c4a-4689-82d6-7912cfefde5b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:42.845501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 51/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:42 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=f6cd27fa85264ac58298f2d2b0667bc7&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:44.008502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e928a-ddb4-4de8-8514-771cabcbeca3 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.009403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e928a-ddb4-4de8-8514-771cabcbeca3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.009871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e928a-ddb4-4de8-8514-771cabcbeca3 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.011828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 47/98] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:44.017146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8222f04-52e5-460e-a72d-14f4d68054ef None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.018014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8222f04-52e5-460e-a72d-14f4d68054ef None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.018533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8222f04-52e5-460e-a72d-14f4d68054ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.040516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f8222f04-52e5-460e-a72d-14f4d68054ef None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:44.042071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f8222f04-52e5-460e-a72d-14f4d68054ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:44.111488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e55f45-decf-4324-b5da-739c933e2a45 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.112374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e55f45-decf-4324-b5da-739c933e2a45 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.112853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e55f45-decf-4324-b5da-739c933e2a45 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.115742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 48/99] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:44.121356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e9b9-8f34-4092-8b2e-6997ba9072da None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.121636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e9b9-8f34-4092-8b2e-6997ba9072da None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.121913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8e9b9-8f34-4092-8b2e-6997ba9072da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.140411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3bf8e9b9-8f34-4092-8b2e-6997ba9072da None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:44.141592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3bf8e9b9-8f34-4092-8b2e-6997ba9072da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:44.145032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 52/100] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:44.259642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:44.268335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 49/101] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:44.466847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4395-5b20-446a-9e5d-9b03da5711c6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.467260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4395-5b20-446a-9e5d-9b03da5711c6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.467454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4395-5b20-446a-9e5d-9b03da5711c6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.468814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 53/102] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:44.474071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af4be-2b92-4760-add4-ca8976384396 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.474403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af4be-2b92-4760-add4-ca8976384396 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.474738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af4be-2b92-4760-add4-ca8976384396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.498138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-409af4be-2b92-4760-add4-ca8976384396 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:44.499338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-409af4be-2b92-4760-add4-ca8976384396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:44.585026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 50/103] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:44.622727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4395-5b20-446a-9e5d-9b03da5711c6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:44.670026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af4be-2b92-4760-add4-ca8976384396 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:44.777696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:44.787536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4395-5b20-446a-9e5d-9b03da5711c6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:44.824764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af4be-2b92-4760-add4-ca8976384396 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:44.846315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:44.856537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:44.857488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.858111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.858570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.860254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:44.861349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:44.862048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:44.862393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:44.862680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:44.862776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:44.871127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 54/104] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:44.875690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:44.879594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:44.884941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:44.885780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:44.886553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:44.888786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 51/105] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:44.894173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.061031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45147-0944-436b-874c-024e5fef5aff None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.112743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a0e15-7cf7-4b51-be15-c206dad25090 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.125409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.126132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.126488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.126838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.127606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:45.127978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.129432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.146464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 52/106] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:45.147328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.159322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.160227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.160680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.161700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.162602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.164298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.218496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:45.278775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 55/107] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:51:44 2022] POST /identity/v3/projects => generated 302 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:45.280027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.294055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e4295-103e-427b-87f5-fe183b218032 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.329937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:45.352161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.353095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.353551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.354085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.367422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d6b561e5-75ba-4135-8444-15811f0cb234 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:45.373412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:45.375085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:45.375888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:45.382812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 53/108] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:45.384639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4457502-c59e-4bc5-9252-be9c2a28d0b3 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.385068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4457502-c59e-4bc5-9252-be9c2a28d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.385508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4457502-c59e-4bc5-9252-be9c2a28d0b3 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.387085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 54/109] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:45.388559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4457502-c59e-4bc5-9252-be9c2a28d0b3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.503652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4457502-c59e-4bc5-9252-be9c2a28d0b3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.507460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:45.539015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f417118-bb7c-4721-b916-c85b3031eb04 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.547164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.548443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.548870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.549282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.550096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:45.550774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.552889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2441a-c109-48c7-872b-ec3fe0615b28 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.564297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 55/110] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:45.565983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.566439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.566855 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.590825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:45.592492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:45.594815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.596119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.596119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.596119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] PATH_INFO: `/v3/users/036308bcad8648e2b9a2284e8ee03a73` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.612629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] RBAC: Authorizing `identity:get_user(user_id=036308bcad8648e2b9a2284e8ee03a73)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.614173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.617811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 56/111] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/users/036308bcad8648e2b9a2284e8ee03a73 => generated 289 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:45.619186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.682771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 56/112] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:45.684127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.738620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:45.764082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72214793-7cfb-4315-80f1-975373837d1f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.800254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.801072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.801423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.801837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.803588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:45.803823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.805878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f03a0f-f88f-4d0e-9ba2-78e2b1c73f72 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.807670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 57/113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:45.808937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4397b-f690-458b-b475-bd7bf264d1dc None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.809325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4397b-f690-458b-b475-bd7bf264d1dc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.809680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4397b-f690-458b-b475-bd7bf264d1dc None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.810997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 58/114] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:45.811809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db989-0c0f-4965-b1ea-70564ca80a48 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:45.816003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4397b-f690-458b-b475-bd7bf264d1dc None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:45.855893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:45.856781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:45.857216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:45.857627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:45.937938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:45.939654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d399cd9-6199-488c-9ee1-e54c810f7a29 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:45.996765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4397b-f690-458b-b475-bd7bf264d1dc None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.046583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.054727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 57/115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:46.056549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.056984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.057404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.058380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.059016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.059416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.059702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.060862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:46.062498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db666c5a-a0c5-478d-9ea2-a31397af9653 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:46.064260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 59/116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:45 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:46.068534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e677e-8367-434f-99bd-692a300f6e45 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.068963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e677e-8367-434f-99bd-692a300f6e45 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.069375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e677e-8367-434f-99bd-692a300f6e45 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.070916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 60/117] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:46.072642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e677e-8367-434f-99bd-692a300f6e45 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.084481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:46.086202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:46.100332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.157557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 58/118] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:46.158398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.207408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e677e-8367-434f-99bd-692a300f6e45 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.244727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.245187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.245528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.245811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.246981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:46.248556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9570ff1-0cfb-4d10-aaef-8c1ed375bcf7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:46.250426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 61/119] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:46.251642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.251932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.252210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.275382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:46.277041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:46.280268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636975cb-638b-48f0-9de8-6a62da7ed0cc None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.307785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.322612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.323234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.323600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.323880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/users/78b783de521c4bc3ade218e12aef00c7/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.326015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, user_id=78b783de521c4bc3ade218e12aef00c7, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:46.327604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:46.357354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 62/120] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:46.358623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.376317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.406430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 59/121] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:51:46 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/users/78b783de521c4bc3ade218e12aef00c7/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:51:46.410041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.495662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8546-0eac-40f6-a3e6-ba28ca2895ba None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.540050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.540707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.540988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.541261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.564587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:46.570831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:46.571835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:46.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:46.574809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 63/122] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:46.575788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.628961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c81e9-474b-45a3-845f-41eaca922cbf None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.665254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.676028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.676642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.677767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.678183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.679403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=036308bcad8648e2b9a2284e8ee03a73) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:46.683799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:46.683799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:46.688861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 60/123] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/role_assignments?user.id=036308bcad8648e2b9a2284e8ee03a73&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:46.689848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.725721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbb17-eb4a-40fd-ae94-40db13a4d349 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.751917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:46.782437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.787076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.787076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.787076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.802287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:46.808416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:46.809383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:46.810467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:46.811182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 64/124] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:46.815140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:46.868613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e610-8586-4db1-a6a8-d00897a1091e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:46.927677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:46.928938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:46.929227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:46.929533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:46.930436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:46.930745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:46.932260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:46.942088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 61/125] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:46.943977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.022356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e3a38-e617-4b1a-aa54-f634cf5b284b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.063313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756c144-4262-4087-9f0e-a06f99baa9e4 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.083679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.095285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.107340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.107981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.108261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.108532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.129292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:47.139021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:47.140011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:47.140819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:47.142598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.143830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.144580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.145052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.146166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:47.146803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.148224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 65/126] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:47.149972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.151024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.171090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 62/127] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:51:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.172333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.359307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a3123-3768-41b4-8b61-4be69dd1cf2f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.373925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914502-2b8a-44da-869a-a4405833af09 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.405159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.405827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.406057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.406335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.407003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:47.407362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.408757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.418007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 63/128] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.418657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.461036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.463815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.463815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.463815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.463815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.465342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.466883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.474950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 66/129] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.476669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.567305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9d274-13db-416a-8e42-1a39bebcfb3c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.611502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.632842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.633498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.633790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.634132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.635774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.637237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.639018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 64/130] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.639642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.668246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914bb00-f854-4ba9-a57d-2aaf7bb0b674 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.719424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.720068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.720878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.721171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.722586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.724554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.726976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 67/131] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.731328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.792476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57ee3-c891-439c-845b-e1a130bc3e28 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.795027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.837117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.838033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.838584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.839031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.840440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.842167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4083dd-0bda-4f90-82a3-95db5fda1d93 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.844163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 65/132] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.845362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e02f7-bfc5-40fc-9cee-a0f882c76b49 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.845851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db481b-18e9-4aa9-ac47-1f8bdb226162 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.846325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db481b-18e9-4aa9-ac47-1f8bdb226162 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.846972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db481b-18e9-4aa9-ac47-1f8bdb226162 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.849362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 66/133] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:47.850280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db481b-18e9-4aa9-ac47-1f8bdb226162 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.876671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:47.885881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:47.886530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:47.889233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:47.889233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:47.889233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:47.890049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:47.891822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 68/134] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:47.892428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:47.964502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db481b-18e9-4aa9-ac47-1f8bdb226162 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:47.997453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eed2b-397c-49d1-b33e-a40f43b12dd4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:48.000145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.003223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.003223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.003223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] PATH_INFO: `/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.003223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] RBAC: Authorizing `identity:get_project(project_id=f6cd27fa85264ac58298f2d2b0667bc7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.004736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deaf933-53be-4fe3-add7-90dcbef27a86 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.012556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 67/135] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:47 2022] GET /identity/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:48.014329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.014912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.015447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.035975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.036488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:48.036934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.037382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.037669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/users/d67b28f7eb04449ba5b42d28ffe55408/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.038607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:48.039683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, user_id=d67b28f7eb04449ba5b42d28ffe55408, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.041206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.071755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:48.108946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 68/136] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:48.112372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:48.154948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 69/137] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:51:47 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/users/d67b28f7eb04449ba5b42d28ffe55408/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:51:48.156196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:48.215777 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3a626-12f7-46f1-92ab-a1a469472595 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:48.258053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.259889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.260167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.260440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/users/d67b28f7eb04449ba5b42d28ffe55408/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.264939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, user_id=d67b28f7eb04449ba5b42d28ffe55408, role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.266390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.277226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:48.345718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 69/138] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:51:48 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/users/d67b28f7eb04449ba5b42d28ffe55408/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:51:48.351814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e51249-55c1-4221-b00a-3a6f6dde1865 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:48.400920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.401688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.401973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.402298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] PATH_INFO: `/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7/users/d67b28f7eb04449ba5b42d28ffe55408/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.410709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6cd27fa85264ac58298f2d2b0667bc7, user_id=d67b28f7eb04449ba5b42d28ffe55408, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.410709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.500849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 70/139] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:51:48 2022] PUT /identity/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7/users/d67b28f7eb04449ba5b42d28ffe55408/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:51:48.534537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:48.574744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:48.673381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab9ad-f60c-4c8f-8918-e8c0b464a5bb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:48.715994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.716651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.716979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.717254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.718466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.720019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdcd3f-9502-495c-9b85-d82f2df05069 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.722997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 70/140] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:48.727713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:48.797300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e987a-54ff-46ea-8644-81c5627c524b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.797615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e987a-54ff-46ea-8644-81c5627c524b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.797862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e987a-54ff-46ea-8644-81c5627c524b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.799379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 71/141] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:51:48.806200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeba2a-db9f-4d28-9e91-f2b02db886a9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.806484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeba2a-db9f-4d28-9e91-f2b02db886a9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.806759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aeba2a-db9f-4d28-9e91-f2b02db886a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.840102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-77aeba2a-db9f-4d28-9e91-f2b02db886a9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:48.841285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-77aeba2a-db9f-4d28-9e91-f2b02db886a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:48.868154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bc2a-cf86-4b17-8335-be7ca5030fc7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:48.899131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:48.913587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:48.914207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:48.914495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:48.914841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:48.915528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:48.917068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f558-8b5b-45e1-bdad-026fda36829c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:48.994106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 72/142] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:49.029229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 71/143] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:51:48 2022] POST /identity/v3/users => generated 283 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:49.198314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cab14-bfe1-4e07-8d48-1867317b2252 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:49.198673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cab14-bfe1-4e07-8d48-1867317b2252 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:49.198873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cab14-bfe1-4e07-8d48-1867317b2252 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:49.200443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 73/144] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:49.206061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:49.206352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:49.206624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:49.230436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:49.231691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:49.321798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 72/145] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:49.398315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cab14-bfe1-4e07-8d48-1867317b2252 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:49.430791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:49.525672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cab14-bfe1-4e07-8d48-1867317b2252 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:49.570696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:49.571353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:49.571633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:49.571954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:49.589119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:49.593936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:49.594788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:49.595579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 74/146] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:49.602997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:49.700169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:49.757801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c333ea0-0eee-48df-9a0e-f578c0ca01c9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:49.759524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:49.806043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:49.825258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb585-ccc7-4a12-b5e9-02a26233422a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:49.849267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:49.849876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:49.850155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:49.850419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:49.855681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:49.855681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:49.855681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:49.868804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:49.869096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 73/147] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:49.869911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:49.870468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:49.870951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:49.874329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:49.886238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:49.892549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:49.893417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:49.894264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:49.897246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 75/148] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:49.908381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.024245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903066-487e-44d8-b6e2-25d98e829c99 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.032532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46372-3f2b-40a7-96bf-0309162ae6d5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.065007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.065768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.066319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.066433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] PATH_INFO: `/v3/users/78b783de521c4bc3ade218e12aef00c7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.069433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] RBAC: Authorizing `identity:get_user(user_id=78b783de521c4bc3ade218e12aef00c7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.071302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.072568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.073394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.073653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 74/149] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:49 2022] GET /identity/v3/users/78b783de521c4bc3ade218e12aef00c7 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:50.073966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.074300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.074976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:50.075355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.076677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.078743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.087452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 76/150] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:50.092946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.105599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:50.113320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:50.213644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee330526-e0a4-4007-8c42-ed6960be0427 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.261037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20553b-b359-4fcb-bfdd-461eda2e6a8e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.283188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.283933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.284223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.284497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] PATH_INFO: `/v3/users/036308bcad8648e2b9a2284e8ee03a73` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.291137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] RBAC: Authorizing `identity:get_user(user_id=036308bcad8648e2b9a2284e8ee03a73)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.291137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.291137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 77/151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:50 2022] GET /identity/v3/users/036308bcad8648e2b9a2284e8ee03a73 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:50.297495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.315816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.316668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.317110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.317572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.406362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.408187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.444270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:50.451164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:50.513341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcbbce-fad7-4319-a072-eae25f35e706 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.529860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 75/152] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:51:50.535430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.591375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.592292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.592727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.593141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.594486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.596214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.598192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 78/153] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:50 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:50.604889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.662904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2a583-f4fa-4f73-b1d2-5bf6ded8f1c6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.742245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b6273-bf57-4adc-9ba7-73f547ee21e0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.760051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:50.772380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.773031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.773482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.773587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.779820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.781300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.782995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 76/154] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:50 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:50.789009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:50.814622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.815565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.816030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.816449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/users/036308bcad8648e2b9a2284e8ee03a73/roles/5d01151b08dd4e3e9c83992195eaa20d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.819348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, user_id=036308bcad8648e2b9a2284e8ee03a73, role_id=5d01151b08dd4e3e9c83992195eaa20d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.820989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c6202-98c0-4285-b6cc-cf4d8fad466d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.912554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 79/155] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:51:50 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/users/036308bcad8648e2b9a2284e8ee03a73/roles/5d01151b08dd4e3e9c83992195eaa20d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:51:50.918003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2d4a8-766b-41b8-b8a6-ec1a2d3e28ad None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:50.956607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:50.957279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:50.957561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:50.957889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:50.958876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=78b783de521c4bc3ade218e12aef00c7) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:50.959292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:50.960744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1fffc-1281-4970-9380-36520b1da1f9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:50.968160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 77/156] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:50 2022] GET /identity/v3/role_assignments?user.id=78b783de521c4bc3ade218e12aef00c7&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:51.624270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de067-3a20-49f6-a9f7-3a387250464c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:51.625159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de067-3a20-49f6-a9f7-3a387250464c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:51.625620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1de067-3a20-49f6-a9f7-3a387250464c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:51.627289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 80/157] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:51.633663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf250-cf9e-43d6-b87a-db8a20d68811 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:51.633953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf250-cf9e-43d6-b87a-db8a20d68811 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:51.634231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf250-cf9e-43d6-b87a-db8a20d68811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:51.666930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-eafdf250-cf9e-43d6-b87a-db8a20d68811 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:51.668224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eafdf250-cf9e-43d6-b87a-db8a20d68811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:51.741639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:51.750674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 78/158] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:51.951864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5ca6-39cf-44fe-bd2f-85daa6527503 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:51.951864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5ca6-39cf-44fe-bd2f-85daa6527503 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:51.951864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b5ca6-39cf-44fe-bd2f-85daa6527503 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:51.951864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 81/159] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:51.964546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:51.964830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:51.965097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:51.992153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:51.993403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:52.076489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 79/160] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:52.190605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5ca6-39cf-44fe-bd2f-85daa6527503 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:52.195638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:52.322433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b5ca6-39cf-44fe-bd2f-85daa6527503 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:52.363284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:52.363928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.364235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.365137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:52.383714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-8696ebb5-f83d-4ef7-ae8c-3702e3803a0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:52.389676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:52.390533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:52.391441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:52.392059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 82/161] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:51:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:52.397812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:52.500111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9361d-01c2-4385-9fb2-742b6903de28 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.500427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9361d-01c2-4385-9fb2-742b6903de28 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.500660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9361d-01c2-4385-9fb2-742b6903de28 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:52.502017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 83/162] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:52.517290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.517572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.517845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:52.554542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:52.555826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:52.586178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ff1da-a085-4e97-b178-1297d97b711e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:52.632095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:52.654003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:52.654927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.655871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.655871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:52.657291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:52.657811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:52.659507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:52.674000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:52.679285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 80/163] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:51:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:52.687397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:52.693301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 84/164] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:52.697011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:52.863503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76c308-4449-4655-a34c-cb266f5a97ef None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:52.905114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:52.906028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.906458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.906880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:52.908564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:52.910368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:52.912912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 81/165] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:52 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:52.915271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192608e3-4d09-4ba2-8658-b64c5f67d205 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:52.918810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:52.984342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:52.988037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:52.988680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:52.988959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:52.989752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.021242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:53.026488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:53.027507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:53.028467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 85/166] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:51:53.053735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.077590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c1ce-28be-400f-a743-e29590d43ca0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.136107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.136761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.137051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.137430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.143167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:53.221893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280f759-c48e-45da-a36e-d4ddd3285272 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.251140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:53.252688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:53.290700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.291341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.291619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.291888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.292560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:53.292859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:53.294239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:53.321235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 86/167] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:53.322728 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.351482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:53.409265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 82/168] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:51:53.410129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.526726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd42a3-bba8-4083-a80a-a23ff7e9f999 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.545984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:53.578458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.579267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.579622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.580002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.593408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-243c3d97-989b-43a0-aa11-7c236b012450 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:53.600499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:53.601403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:53.602200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 87/169] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:53.603007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18684-50e4-452c-a4c9-7c79212636e8 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.603574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18684-50e4-452c-a4c9-7c79212636e8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.603670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18684-50e4-452c-a4c9-7c79212636e8 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.605247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 88/170] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:53.605960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af18684-50e4-452c-a4c9-7c79212636e8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.723324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af18684-50e4-452c-a4c9-7c79212636e8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.724118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a1fd1-d534-4d45-91b2-269f6e03d1d8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.763732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.764837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.765389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.765818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.766985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:53.769158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:53.771643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 89/171] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:53.772793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.798263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.799490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.799788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.800061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.801556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:53.803824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bdbf4-8fcd-4577-b669-a0b6db02ee25 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:53.806464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 83/172] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:53.811850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5e0-48c6-44e4-881b-c65a32942376 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.812251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5e0-48c6-44e4-881b-c65a32942376 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.812587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5e0-48c6-44e4-881b-c65a32942376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.843131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-31dca5e0-48c6-44e4-881b-c65a32942376 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:53.843946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:53.845392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-31dca5e0-48c6-44e4-881b-c65a32942376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:53.866806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:53.903898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86584549-ef86-42c6-8fd7-f4f5f61bbeb9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:53.924890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 84/173] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:53.929017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d893-7c34-42a8-b571-80ddbc4a951e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.929596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d893-7c34-42a8-b571-80ddbc4a951e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.929880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d893-7c34-42a8-b571-80ddbc4a951e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.931420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 85/174] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:53.935860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d893-7c34-42a8-b571-80ddbc4a951e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:53.952562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:53.953178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:53.953454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:53.953715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:53.954826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:53.955190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:53.956921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:53.969517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 90/175] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:53.970739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.070118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d893-7c34-42a8-b571-80ddbc4a951e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.090771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d34ce-5a3b-457d-84b1-fc3a46c43d3d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.099139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.109353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:54.114888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6a425-1541-47ff-9ecd-661231b89cfa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:54.121881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 86/176] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:54.127567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812004b-6c13-4525-8750-ba2375e28104 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.128523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812004b-6c13-4525-8750-ba2375e28104 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.129003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812004b-6c13-4525-8750-ba2375e28104 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.130829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 87/177] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:54.138392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.140244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.141089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.145268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.146330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.146794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.147267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:54.182441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:54.198415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.278278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 88/178] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:54.279621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.296580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:54.298418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3768a22-3618-4025-8b6f-9250e0a5025f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:54.423746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 91/179] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:54.425121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569485f-f72f-45fe-b860-8efd9e023800 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.425476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569485f-f72f-45fe-b860-8efd9e023800 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.425819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569485f-f72f-45fe-b860-8efd9e023800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.426633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde73db2-3e5c-4d4d-9a7b-d41bfcae4f09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.438798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.446261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d569485f-f72f-45fe-b860-8efd9e023800 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:54.447500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d569485f-f72f-45fe-b860-8efd9e023800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:54.474554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.475225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.475496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.475764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] PATH_INFO: `/v3/users/d67b28f7eb04449ba5b42d28ffe55408` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.477118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] RBAC: Authorizing `identity:get_user(user_id=d67b28f7eb04449ba5b42d28ffe55408)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:54.479036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:54.480834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 89/180] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:54 2022] GET /identity/v3/users/d67b28f7eb04449ba5b42d28ffe55408 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:54.481611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.504915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.539597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 92/181] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:54.540914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91bdd0-febd-456f-be19-fb6f1f92e525 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.541200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91bdd0-febd-456f-be19-fb6f1f92e525 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.541471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91bdd0-febd-456f-be19-fb6f1f92e525 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.542695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 93/182] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:54.543402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91bdd0-febd-456f-be19-fb6f1f92e525 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.629055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbba73-3342-47a8-b866-79f85beeebef None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.664065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.664696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.664966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.665229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.672035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91bdd0-febd-456f-be19-fb6f1f92e525 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.676078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:54.680991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:54.681886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:54.682700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 90/183] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:54.683416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.716781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.717905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.718502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.718836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.720041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:54.721470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a75d-4069-435c-8652-8eeb98fa0f65 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:54.727413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 94/184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:54 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:54.728499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.728780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.729053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.736518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.750892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:54.752123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:54.776627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.804522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74574241-bb52-44d2-b75b-effdb7549c8f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.826386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 95/185] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:51:54.827265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:54.844015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.844620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.844892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.845161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.922693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:54.924210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:54.944693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacfd25-5e77-4764-83e9-ab641a992301 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:54.980764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:54.981760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:54.982313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:54.982782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:54.985363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:54.996180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:55.001440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:55.002330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 96/186] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:51:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:55.003909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.015847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.041953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 91/187] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:55.042943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.112133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacfcd-10b4-48c3-99d8-ea0f4065d5c8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.143625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcb096-c855-46d3-88cd-60873d003dfc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.147940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.148542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.148822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.149095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.149750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:55.150050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.151468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.159962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 97/188] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.160690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.186681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.187309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.187586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.187852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.188566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:55.188864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.190374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.196773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 92/189] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.197480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.218181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.258713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.279683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bebd43-558b-4c14-bc5c-9b43d76f6549 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.317226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.318480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.318778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.319157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.320339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.322062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.328331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 98/190] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.328331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.332116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57a1db-c93c-4f09-92a5-4c20fff078c1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.372000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.373029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.373319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.373596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.374261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:55.374565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.376688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.385595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 93/191] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.386673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.444086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b21c77-f11b-4e65-a915-b755faabe42a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.467243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.477722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.478415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.478700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.482405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] PATH_INFO: `/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.482405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] RBAC: Authorizing `identity:get_project(project_id=f6cd27fa85264ac58298f2d2b0667bc7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.482405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.483250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 99/192] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/projects/f6cd27fa85264ac58298f2d2b0667bc7 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.488219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.488482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.491962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dc3e1-f96a-433c-88df-ff8aad78854f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.525082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.525699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.525974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.526237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.537197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:55.542546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:55.543667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 94/193] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:55.544668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.596517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3655a6-bcf6-4009-9c0b-e19f4e62f1ea None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.629268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.630403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.631228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.631506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.645170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13155-374e-4df8-80cd-0e7f17afbf4c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.647144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.648587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.650189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 100/194] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.650954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.680826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.681423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.681697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.681965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.682604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.684066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.699540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.748649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31c5b3-7a5b-4d3c-9377-9d46b083d713 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.781653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.782275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.782554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.782825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.783970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.785420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.787181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 101/195] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.787944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.799192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 95/196] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:51:55 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:51:55.800592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.821408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.907046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a43155-60ea-4305-8b8e-3ec4b66aa3f5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.918908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa49dd-95e2-43a5-9b4c-465e87b26952 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:55.927293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:55.946698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.947086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.947410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.949729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.949729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=f6cd27fa85264ac58298f2d2b0667bc7, user.id=d67b28f7eb04449ba5b42d28ffe55408) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:55.949729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.951590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.956510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 102/197] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/role_assignments?user.id=d67b28f7eb04449ba5b42d28ffe55408&scope.project.id=f6cd27fa85264ac58298f2d2b0667bc7&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.957553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:55.959216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:55.959921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:55.960209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:55.960481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:55.961159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:55.961458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:55.962896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:55.977078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 96/198] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:55.977078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.092706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.112381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541910-16cb-4888-b1c9-7bea1546d6c6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.128542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e2098-4208-418c-bafd-ee68d7da9bc8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.156805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.157445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.157702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.158255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.186734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.187372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.187644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.187911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.222837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.251433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.253568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.279722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.281727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.363545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 103/199] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:56.364152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.389219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.423937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 97/200] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:56.424623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.458598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.477564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146e00a-b1b2-4c26-836a-01c2b94bd9ef None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.512178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.512869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.513144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.513412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] PATH_INFO: `/v3/users/036308bcad8648e2b9a2284e8ee03a73` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.514544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] RBAC: Authorizing `identity:get_user(user_id=036308bcad8648e2b9a2284e8ee03a73)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.515998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.517882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 104/201] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/users/036308bcad8648e2b9a2284e8ee03a73 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:56.518525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.550561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef3855-8742-4076-8abb-1766c2b43ddf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.599607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.600746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.601239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.601727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.603264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.605842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.608658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 98/202] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:56.609605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.651497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668c76-6ac7-4103-93a9-7aafd05551dc None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.662242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.699710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.700624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.701066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.701538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.702901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.704560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.706372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 105/203] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:56.707309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.722048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46bfe6-4305-43cc-baf7-9002cb9011c4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.728243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:56.766057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.767199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.767796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.768307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.820856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141b084-5abf-4758-94a2-f3aab05c7ec6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:56.849798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.851086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.869153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:56.870257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:56.871138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:56.871598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:56.872582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=d03c3e24f4414f57b65bbb5525c924ea) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:56.873081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:56.876513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:56.891151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 106/204] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/role_assignments?user.id=d03c3e24f4414f57b65bbb5525c924ea&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:56.891151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:56.927572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:57.032916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 99/205] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:51:57.043838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:57.065193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:57.089898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb8429-12a9-4bbc-aa9e-9192a9a3b709 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:57.128955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:57.129567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:57.130394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:57.130797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:57.131622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=d03c3e24f4414f57b65bbb5525c924ea) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:57.131976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:57.133496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:57.139147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 107/206] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:56 2022] GET /identity/v3/role_assignments?user.id=d03c3e24f4414f57b65bbb5525c924ea&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:57.171601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75649a3-e966-415d-8c79-2e6f7acc9de3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:57.215175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:57.216596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:57.217110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:57.217597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:57.219281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:57.221108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c269d-d101-4d5f-8aea-4534bcc38085 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:57.223689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 100/207] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:57 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:57.231142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:57.383744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f12729-8447-4b3b-aa76-5ae1f308b0bb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:57.424597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:57.425207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:57.425486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:57.425757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:57.426476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, scope.project.id=3bee042dbad049779321aede9dedec4e, user.id=036308bcad8648e2b9a2284e8ee03a73) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:57.427340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:57.428865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e86415-7dfb-4983-ae85-57a286ec279f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:57.435026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 108/208] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:57 2022] GET /identity/v3/role_assignments?user.id=036308bcad8648e2b9a2284e8ee03a73&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:58.083470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db342d8-bc0a-4fad-ab55-9d981aadfdf6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.084629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db342d8-bc0a-4fad-ab55-9d981aadfdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.085095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db342d8-bc0a-4fad-ab55-9d981aadfdf6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.086756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 101/209] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:58.092202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.093007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.093489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.112351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858633-0ba5-4359-bce5-7e1d6569df3f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.112351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858633-0ba5-4359-bce5-7e1d6569df3f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.112351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858633-0ba5-4359-bce5-7e1d6569df3f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.113716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 102/210] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:58.119333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.119616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.119888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.120464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:58.121852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:58.128987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:58.144099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:58.146109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:58.156441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:58.199224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 109/211] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:58.221869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 103/212] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:58.470385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:58.505291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:58.597473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95992305-ac15-4763-960c-ca8cb709cb33 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:58.638139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:58.638772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.639046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.639346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.644283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a6e01-4d7a-4fb6-a396-e8ad25484010 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:58.648204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:58.651244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:58.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:58.657204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:58.658075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 110/213] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:58.662742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:58.697955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:58.699224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.699740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.700446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.702402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:58.704323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:58.712546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 104/214] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:51:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:58.713130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:58.735617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:58.778569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887227-d0bd-4d69-88a3-9861f025bda1 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:58.815287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:58.815892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.816165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.816432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.818026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:58.818026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:58.819667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:58.828695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 111/215] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:51:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:58.834481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:58.858222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf36d4-bf90-4552-8c90-58b57785001b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:58.909554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:58.915322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:58.920938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.920938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.920938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:58.920938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:58.920938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24c499-420e-42eb-bdeb-41020a5cd0cc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:58.951640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc9011-656e-442c-810a-a5616e07ee5e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:58.988559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:58.989118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:58.989423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:58.989721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] PATH_INFO: `/v3/users/345149d6c19c4b3b829d18b4b3b69126` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.004419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] RBAC: Authorizing `identity:get_user(user_id=345149d6c19c4b3b829d18b4b3b69126)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:59.006052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:59.007773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 112/216] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:58 2022] GET /identity/v3/users/345149d6c19c4b3b829d18b4b3b69126 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:59.013537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.036053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 105/217] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:51:58 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:51:59.060153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db8f87-84ea-4424-94f3-222cca754d4b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.060483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db8f87-84ea-4424-94f3-222cca754d4b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.060692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db8f87-84ea-4424-94f3-222cca754d4b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.062191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 106/218] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:51:59.069353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.069629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.069972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.090352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:59.091628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:59.153767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157de14-f7e0-47e7-b0c2-f74455f36453 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:59.162797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:59.168695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 107/219] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:59.179454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:59.194132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:59.194747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.195144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.202782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.272188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:59.273686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed791d-48c0-4476-8904-506993a80433 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:59.380129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 113/220] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:51:59.386111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.405401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51e3f-a142-464d-8799-a233dae57bb2 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.405717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51e3f-a142-464d-8799-a233dae57bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.405959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b51e3f-a142-464d-8799-a233dae57bb2 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.407716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 114/221] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:51:59.413613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.415339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.415339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.434971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:51:59.436719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:51:59.454932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:59.520930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a5ac8-ebff-4275-baf3-8d2aef4e6b3c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:59.522718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 115/222] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:51:59.524032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.565854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:59.565854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.565854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.565854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.567204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:59.569729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:59.571563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 108/223] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:51:59.582557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.660363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584044b-4fd8-42f4-ba64-a88ea03028e5 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:59.701660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:59.702301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.702572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.702839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.713652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:51:59.718772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:51:59.719703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:51:59.720504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:51:59.722857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 116/224] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:51:59.728515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.761926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:59.779577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:51:59.847840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8f25a-7260-4f57-90dc-0ec5db9e9ff9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:59.855178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0b3f7-e9c8-4e48-a522-79ef16b04a84 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:51:59.891981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:59.893023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.893434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.893855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.894712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=345149d6c19c4b3b829d18b4b3b69126) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:59.895184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:59.895763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:51:59.896447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:51:59.896730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:51:59.897005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:51:59.897674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:51:59.897967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:59.898076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:51:59.899488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:51:59.904280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 109/225] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/role_assignments?user.id=345149d6c19c4b3b829d18b4b3b69126&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:59.905373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:51:59.924012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 117/226] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:51:59.931022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.069745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b4c24-6d35-4c82-acb4-4b96854b9180 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.100115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.118887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.126063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.126972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.127475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.127927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.132887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b521e-be00-4240-a2d7-0fdfff0d4b3f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.139194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:00.146410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:00.147632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:00.148458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:00.150599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 110/227] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:00.157030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.170250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.170893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.171194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.171471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.172609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.174046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.176059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 118/228] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:51:59 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:00.182276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.278859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac48d3-b6b6-4833-b92f-278c9af97ee7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.285232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839f775-6c77-486b-a997-dbba6cf3503d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.318581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.319683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.320253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.320545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.320915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.321308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.321660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.322004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.322393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:00.322867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.323294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.324692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.324807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.331208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 119/229] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:00 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:00.332358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.334652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 111/230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:00.339818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.345451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.401306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.438936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5497166-9865-4ca9-819e-fe48f9943527 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.462451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b4991-c49b-4669-82a6-c5363e67d9eb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.476571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.477350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.477750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.478096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/d03c3e24f4414f57b65bbb5525c924ea/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.484402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, user_id=d03c3e24f4414f57b65bbb5525c924ea, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.486093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.513354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.515582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.515582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.515582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.516507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.518227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.520353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 112/231] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:00 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:00.528460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.577763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 120/232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:00 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/d03c3e24f4414f57b65bbb5525c924ea/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:52:00.641436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedd8aa-10dc-4fe9-b21e-185693f02c63 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.664239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.679063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.679722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.682797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.682797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.682797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.683023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2d7ca-8e7c-4c96-8eec-8160d877e266 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.684717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 113/233] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:00 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:00.690690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:00.780806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:00.826023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72234602-c01f-41c7-8374-bf63c77dfd26 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:00.863621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:00.864268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:00.864576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:00.864849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/d03c3e24f4414f57b65bbb5525c924ea/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:00.867323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, user_id=d03c3e24f4414f57b65bbb5525c924ea, role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:00.868767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c8d2b-f2e3-4c82-8e07-183a70fa95be None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:00.983389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 121/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:00 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/d03c3e24f4414f57b65bbb5525c924ea/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:01.450562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68a34-ed99-4b91-823a-315790fbd12d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:01.451078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68a34-ed99-4b91-823a-315790fbd12d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:01.451078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68a34-ed99-4b91-823a-315790fbd12d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:01.452473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 114/235] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:01.457733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f58f-14d5-4495-a962-e74d13de2d19 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:01.458385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f58f-14d5-4495-a962-e74d13de2d19 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:01.458501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218f58f-14d5-4495-a962-e74d13de2d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:01.482624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5218f58f-14d5-4495-a962-e74d13de2d19 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:01.483833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5218f58f-14d5-4495-a962-e74d13de2d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:01.578056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 122/236] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:01.625906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b696be-01b2-45b6-b74b-78faaf2ea8a9 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:01.626214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b696be-01b2-45b6-b74b-78faaf2ea8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:01.626478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b696be-01b2-45b6-b74b-78faaf2ea8a9 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:01.628019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 115/237] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:01.633776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:01.634066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:01.634341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:01.660077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:01.661776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:01.735607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:01.757373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 123/238] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:01.909978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b696be-01b2-45b6-b74b-78faaf2ea8a9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.036691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b696be-01b2-45b6-b74b-78faaf2ea8a9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.085624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.086257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.086536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.086807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.087952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:02.089463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:02.091055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 116/239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:02.096637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.131460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.197309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:02.251221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bd132-a182-48fa-a093-d6c1de6bb896 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.270054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3392-b1c2-4f47-964a-a3ebcffeb7a5 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.308408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.308959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.309236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.309506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.339734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:02.343651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.344256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.344528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.344792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:02.349961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:02.350867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:02.351719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 117/240] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:02.352399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:02.352935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c2761-2bf0-44aa-ad85-3988280051ad None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:02.361618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.444040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 124/241] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:02 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:02.445949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0261b4f-35da-462e-afd6-ac3c65bdda79 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.446496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0261b4f-35da-462e-afd6-ac3c65bdda79 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.446770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0261b4f-35da-462e-afd6-ac3c65bdda79 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.449601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 125/242] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:02.461605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.463574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.463574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.496420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:02.497641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:02.525808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e7d3-730d-4d97-ad8d-e5588c184afb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.567091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:02.592536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:02.609271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.609889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.610169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.610442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.611247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:02.611551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:02.612941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:02.626735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 118/243] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:02.633160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.636331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 126/244] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:02.746793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488ff3-917b-4072-b92c-86d00e73fe83 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.782754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.783410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.783692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.784025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] PATH_INFO: `/v3/users/ce9c1f057f224e1cb2364a1248ecb32b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:02.798630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=ce9c1f057f224e1cb2364a1248ecb32b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:02.800197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:02.802489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 119/245] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:02 2022] GET /identity/v3/users/ce9c1f057f224e1cb2364a1248ecb32b => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:02.808566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.926334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994056a5-264f-4c59-ade1-27d9155446fd None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:02.926712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:02.975972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:02.981477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:02.982095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:02.982372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:02.982640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.088745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:03.120419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62930b-2d64-4bf8-923d-c80818d7a5c8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.127304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.128789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.172824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.173462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.173738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.174002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.194305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:03.202582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:03.203968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:03.205961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 120/246] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:03.210618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:03.256362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 127/247] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:03.262179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:03.354175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:03.382901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0559d-3133-43b1-ba61-a07dceaa4cfa None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.394483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:03.407516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0137392-6d9b-4657-bd5c-830322d6fcb1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.419746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.420434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.420710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.420976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.422114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.427539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b0bfc-1be6-4d0a-8513-276af94438dd None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.427539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 128/248] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:03 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:03.427539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a892ab-9072-4a52-8457-721d7e3f97ee None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.427801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a892ab-9072-4a52-8457-721d7e3f97ee None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.428021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a892ab-9072-4a52-8457-721d7e3f97ee None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.429441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 129/249] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:03.434734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.435016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.435313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.455332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:03.456546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:03.525982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 130/250] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:03.526858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:03.527821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.528456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.528740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.529015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.529691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:03.529996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.531492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.557855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 121/251] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:03.563507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:03.662720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:03.679993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2ac8c-d00d-4ec3-a5c9-c48f2fb6a699 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.701217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e4a10-d259-4a64-8efd-30d424ae2c09 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.723325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.723325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.723325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.723325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] PATH_INFO: `/v3/users/345149d6c19c4b3b829d18b4b3b69126` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=345149d6c19c4b3b829d18b4b3b69126)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9db962-9a84-43f3-a9b0-c47bd7516fe3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 122/252] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:03 2022] GET /identity/v3/users/345149d6c19c4b3b829d18b4b3b69126 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479102ec-0390-45db-ad59-be17dd8d74a1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479102ec-0390-45db-ad59-be17dd8d74a1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.724227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479102ec-0390-45db-ad59-be17dd8d74a1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.729036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 123/253] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:03.729840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479102ec-0390-45db-ad59-be17dd8d74a1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:03.736754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.737386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.737629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.737897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.738618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=ce9c1f057f224e1cb2364a1248ecb32b) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:03.738923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.741339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd2ab0-f756-4862-97f0-5e05e59b190b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.747112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 131/254] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:03 2022] GET /identity/v3/role_assignments?user.id=ce9c1f057f224e1cb2364a1248ecb32b&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:03.748796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.749080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.749687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.771493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:03.785870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:03.791362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:03.876363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 132/255] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:03.920500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479102ec-0390-45db-ad59-be17dd8d74a1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:03.971838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:03.972859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:03.973508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:03.974548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:03.975979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:03.977688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:03.979580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 124/256] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:03 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:03.988160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.031651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.081766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.136826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ce22e-6461-403d-99ce-4cad3836690e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.137368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bb212-affe-4587-8c6b-ad9a1ce5ff54 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.139407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.180430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.181082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.181364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.181639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.184792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.185459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.185739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.186010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/345149d6c19c4b3b829d18b4b3b69126/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.188101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, user_id=345149d6c19c4b3b829d18b4b3b69126, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:04.189559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:04.215281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:04.225633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:04.226800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:04.227691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:04.230165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 125/257] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:04.235523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.285271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 133/258] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:03 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/345149d6c19c4b3b829d18b4b3b69126/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:04.336502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.367384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b7973-18a9-4ba2-b5fb-7d1274111cac None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.448822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.449444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.449725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.450001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.450689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:04.450967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:04.452411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:04.466227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 126/259] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:04.467761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.487253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.505735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab531d-466f-4038-860d-99cd8b862f56 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.538024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.556703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.557854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.564273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.565045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.578507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:04.585365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:04.587968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:04.588853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:04.590204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7d4c7-2826-4636-94fd-31d006e8b15c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.590723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 134/260] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:04.596117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.629254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.630032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.630446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.630723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.631893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:04.633699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:04.638024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 127/261] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:04.641558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.733442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.740053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71861354-ec8d-40c1-b388-0c43a3e5a75e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.749765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816d56-a47d-4e8e-85ff-2c554846ff0e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:04.785856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.791243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.791243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.791243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.796628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:04.797482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:04.797950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:04.798508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:04.799412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:04.800015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:04.801583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:04.814975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 135/262] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:04.820416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:04.858389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:04.859910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:04.949621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.957030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:04.971196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 128/263] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:52:04.972972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:05.065746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2671315-2e80-4c84-93d5-9f48452cd3a4 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:05.117598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:05.118244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.118525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.118796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] PATH_INFO: `/v3/users/d03c3e24f4414f57b65bbb5525c924ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.119978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] RBAC: Authorizing `identity:get_user(user_id=d03c3e24f4414f57b65bbb5525c924ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:05.122165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:05.124399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 136/264] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/users/d03c3e24f4414f57b65bbb5525c924ea => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:05.132166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:05.162479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db629-d4d4-4195-b05e-93d0a229aaa1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:05.232202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:05.232836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.233142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.233388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.235514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:05.236972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:05.243928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 129/265] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:04 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:05.249289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:05.328646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03177f2-704b-4a3d-a85a-16a585693be3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:05.351599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:05.361472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:05.404579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:05.405197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.405472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.405744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.462170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21ad6-1c92-433d-a7be-a70bf399a067 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:05.544828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:05.545454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.545737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.546009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.555971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=d03c3e24f4414f57b65bbb5525c924ea) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:05.556293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:05.557846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1758ed0-544c-4989-824e-b70ecce3667d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:05.569524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 130/266] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:05 2022] GET /identity/v3/role_assignments?user.id=d03c3e24f4414f57b65bbb5525c924ea&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:05.596414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:05.597827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:05.760794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6528d-3a3c-4cf9-abad-84691b815591 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.761320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6528d-3a3c-4cf9-abad-84691b815591 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.761320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6528d-3a3c-4cf9-abad-84691b815591 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.762745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 131/267] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:52:05.775418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724ba3-4cee-4519-8665-21cf247ad652 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.775766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724ba3-4cee-4519-8665-21cf247ad652 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.775981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724ba3-4cee-4519-8665-21cf247ad652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.779647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 137/268] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:52:05.786282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:05.808132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5a724ba3-4cee-4519-8665-21cf247ad652 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:05.809353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5a724ba3-4cee-4519-8665-21cf247ad652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:05.820222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:05.919992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 132/269] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:05.927740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfd29e-4f42-4ea6-8213-cb0381e3b62e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.927740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfd29e-4f42-4ea6-8213-cb0381e3b62e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.927740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfd29e-4f42-4ea6-8213-cb0381e3b62e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.927740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 133/270] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:05.934151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:05.934432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:05.934700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:05.954010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:05.955221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:05.997892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ebcc7-2e75-468c-971d-94a7ba358d98 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.015415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.038506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.039235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.039514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.039831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.040959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.042519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.044188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 138/271] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:05 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:06.050026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.077360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 134/272] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:06.144318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.160563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e97c61-5b68-4400-8029-76453546cc79 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.200825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.201740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.202058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.202336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.203211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=d03c3e24f4414f57b65bbb5525c924ea) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:06.203521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.204969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.212086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 139/273] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/role_assignments?user.id=d03c3e24f4414f57b65bbb5525c924ea&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:06.257184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.371178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e1880-74d3-47b3-aee3-e2168decf4a8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.406804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.407858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.408337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.408807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.419646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:06.424813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:06.425696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:06.426504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:06.434385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 135/274] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:06.438298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.506318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.555734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.582370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5520-96a7-4601-b1a6-3d2859949700 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.618567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.619231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.619736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.619736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.620384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:06.620743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.622119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ae8d-2670-4387-8e95-cdb1c0bbcda2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.632522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bd151-caff-49ab-905e-5cd3abedb6cc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.636375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 140/275] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:06.637562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa61-5e3f-4eef-b1ba-667c0d05988b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.637846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa61-5e3f-4eef-b1ba-667c0d05988b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.638118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa61-5e3f-4eef-b1ba-667c0d05988b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.643311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 141/276] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:06.643311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7aa61-5e3f-4eef-b1ba-667c0d05988b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.655698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.673401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.674023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.674299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.674568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.675683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.677378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8ec03-b773-4f74-9637-e0a7ab02da98 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.678962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 136/277] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:06.684479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478490-8cf2-4675-8438-f103826f1629 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.684760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478490-8cf2-4675-8438-f103826f1629 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.685021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478490-8cf2-4675-8438-f103826f1629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.705845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-94478490-8cf2-4675-8438-f103826f1629 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:06.707031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-94478490-8cf2-4675-8438-f103826f1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:06.752264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7aa61-5e3f-4eef-b1ba-667c0d05988b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.781169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 137/278] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:06.783145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94478490-8cf2-4675-8438-f103826f1629 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.787488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.788117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.788396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.788666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] PATH_INFO: `/v3/users/ad9ecb89c2384414925d36e3f4696c54` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.803714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9ecb89c2384414925d36e3f4696c54)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.805263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.807135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 142/279] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/users/ad9ecb89c2384414925d36e3f4696c54 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:06.812690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:06.843622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.889706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:06.925496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3cd53-5b77-4f5c-931f-1ca834caa797 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.932688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94478490-8cf2-4675-8438-f103826f1629 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:06.965112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.965877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.966270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.966634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.967503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, user.id=ad9ecb89c2384414925d36e3f4696c54, scope.system=all) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:06.968009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.969653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:06.977593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 143/280] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 08:52:06 2022] GET /identity/v3/role_assignments?user.id=ad9ecb89c2384414925d36e3f4696c54&scope.system=all&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:06.978342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:06.979131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:06.979539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:06.979935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:06.980734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:06.982366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd4341-aa09-424b-a08e-63abb3ce83ec None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:07.060329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.084838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 138/281] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:06 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:07.170196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b034-49f7-410b-a6cc-9b28c57e1ed6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.170837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b034-49f7-410b-a6cc-9b28c57e1ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.171335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b034-49f7-410b-a6cc-9b28c57e1ed6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.173170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 139/282] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:52:07.180289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.180928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.181284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.197454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc5606-3adc-4aaf-bc84-92c3faf285ec None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:07.201485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:07.202880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:07.234349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:07.242139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:07.249355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.249355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.249355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.257861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:07.264073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:07.265216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:07.266212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 144/283] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:07.271493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.291342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 140/284] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:07.390743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136e606-5e27-48cf-89ed-f8b0e71b05c0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:07.426678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:07.427863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.428215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.428485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.429218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:07.429519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:07.430894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:07.439936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 145/285] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:07.446489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.472321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:07.560048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.574593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5e96-ac29-4e4f-8ec7-15f275f623b0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:07.633061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:07.634040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.634511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.634999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] PATH_INFO: `/v3/users/ce9c1f057f224e1cb2364a1248ecb32b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.636372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] RBAC: Authorizing `identity:get_user(user_id=ce9c1f057f224e1cb2364a1248ecb32b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:07.638018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:07.640456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 141/286] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/users/ce9c1f057f224e1cb2364a1248ecb32b => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:07.646730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.664416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:07.722455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d72c-898a-44b3-87de-f2901d60eddb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:07.767661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:07.777061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b18420-8d52-42a4-b90d-e75cd7b7af1a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:07.803035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:07.804034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.804653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.805513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.816732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:07.817411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:07.817899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:07.818298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:07.821234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:07.822690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:07.824377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 142/287] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:07.828652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:07.832724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:07.836326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:07.837222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:07.838039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 146/288] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:07.843959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:07.984463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fb9e3-2780-42d2-916c-ae9045ef126e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:08.010299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:08.018946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b9e1a-ce41-4a92-aea0-d3a620160ab8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:08.047218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:08.048322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:08.048997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:08.049476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:08.050643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:08.051470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:08.053803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:08.073593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 147/289] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:08.074529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:08.079819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:08.080997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:08.081498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:08.082382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/ce9c1f057f224e1cb2364a1248ecb32b/roles/5d01151b08dd4e3e9c83992195eaa20d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:08.084847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, user_id=ce9c1f057f224e1cb2364a1248ecb32b, role_id=5d01151b08dd4e3e9c83992195eaa20d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:08.086670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:08.173356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 143/290] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:07 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/users/ce9c1f057f224e1cb2364a1248ecb32b/roles/5d01151b08dd4e3e9c83992195eaa20d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:08.195705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf703cc-7bbd-4fdf-9bab-b79d84a6655c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:08.234203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:08.234918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:08.235156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:08.235433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] PATH_INFO: `/v3/users/345149d6c19c4b3b829d18b4b3b69126` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:08.236630 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] RBAC: Authorizing `identity:get_user(user_id=345149d6c19c4b3b829d18b4b3b69126)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:08.238083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:08.240731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 148/291] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:08 2022] GET /identity/v3/users/345149d6c19c4b3b829d18b4b3b69126 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:08.246808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:08.283511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:08.399175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7e27-93e2-4c24-8e59-14923398faa3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:08.439428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:08.440035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:08.440310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:08.440578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:08.512427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:08.513812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:08.535457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:08.633630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 144/292] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:08.639337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:08.719231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:08.802140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187879a3-c5f4-4e00-b817-2dd2ca056261 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:08.852886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:08.853520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:08.853796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:08.854061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:08.859792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:08.861227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a177e56-4af2-4ce1-a581-c1fe0b3f351e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:08.863201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 149/293] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:08 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:08.869418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:09.022985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98c8c-e923-468f-9183-715ad8f145c7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:09.046878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:09.074809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:09.075371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.075649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.075922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.076650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=345149d6c19c4b3b829d18b4b3b69126) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:09.076952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:09.078412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36ac62-4e25-4c5d-94d5-71b824daa1c4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:09.087947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 145/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:08 2022] GET /identity/v3/role_assignments?user.id=345149d6c19c4b3b829d18b4b3b69126&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:09.244287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3cbe-a5f4-4217-9a61-f35f42d0b913 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.244676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3cbe-a5f4-4217-9a61-f35f42d0b913 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.244860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3cbe-a5f4-4217-9a61-f35f42d0b913 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.246429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 150/295] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:09.252118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573c56-d4e2-47f2-8421-ea95130aef82 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.252405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573c56-d4e2-47f2-8421-ea95130aef82 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.252680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573c56-d4e2-47f2-8421-ea95130aef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.280262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-36573c56-d4e2-47f2-8421-ea95130aef82 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:09.281525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-36573c56-d4e2-47f2-8421-ea95130aef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:09.361142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 146/296] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:09.577799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9c20-4091-4434-9f74-1c2624842d38 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.578110 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9c20-4091-4434-9f74-1c2624842d38 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.578348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9c20-4091-4434-9f74-1c2624842d38 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.579978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 151/297] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:09.585386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.585938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.586440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.609008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:09.610547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:09.653952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c9c20-4091-4434-9f74-1c2624842d38 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:09.696537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 147/298] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:09.780642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c9c20-4091-4434-9f74-1c2624842d38 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:09.794693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:09.816567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:09.817873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.817990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.818212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.829058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-542726fa-8f84-4f61-a110-6e9ada876106 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:09.834558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:09.835515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:09.836318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 152/299] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:09.842279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:09.861989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273abd-98c8-4351-bc54-15d80c8dcbbb None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.862275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273abd-98c8-4351-bc54-15d80c8dcbbb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.862532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273abd-98c8-4351-bc54-15d80c8dcbbb None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.864241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 153/300] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:09.871677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:09.872111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:09.872518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:09.892553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:09.893937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:09.895961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:09.975217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cafefb-0422-4f4b-a52e-9cf1572ee3c0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:09.975842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 154/301] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:10.003057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.009423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.010035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.010316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.010591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.011286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:10.011595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.012980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.022785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 148/302] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:10.028314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.109387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4b6b6-b454-4b67-8618-bc510cdf07f7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.110964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.138315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96447-fc7c-4387-8342-9d057ca80ad1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.145530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.146561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.147498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.147498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.148060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.161518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:10.166592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:10.167795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:10.168647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:10.174716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 155/303] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:10.175072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.189833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.190791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.191459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.192040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] PATH_INFO: `/v3/users/ad9ecb89c2384414925d36e3f4696c54` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.193933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] RBAC: Authorizing `identity:get_user(user_id=ad9ecb89c2384414925d36e3f4696c54)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.195791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.197765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 149/304] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/users/ad9ecb89c2384414925d36e3f4696c54 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:10.203715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.283665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae444e8-4bbc-4600-b35c-0613a54734d3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.319788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.320383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.320656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.320921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.321568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:10.321862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.323258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.335269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 156/305] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:10.335269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.337810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48587b-6162-487e-a611-64f2bd562e6e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.366312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.372659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.378135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.378135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.378135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] PATH_INFO: `/v3/system/users/ad9ecb89c2384414925d36e3f4696c54/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.378135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ad9ecb89c2384414925d36e3f4696c54, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.378135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.443831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839339-44f2-49a2-bb8d-592fa189bef9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.477049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.487369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 150/306] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Sep 1 08:52:10 2022] PUT /identity/v3/system/users/ad9ecb89c2384414925d36e3f4696c54/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:52:10.487574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.487574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.487574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.487834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.488941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.491327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.493201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 157/307] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:10.499039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.530396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.607182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.617838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67d44e-6fb4-46d4-bbb5-981cedaf8698 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.621116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774df1b-7329-47dc-8d9f-296310e99b5d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.657643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.657973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.658278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.658529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.658651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.658827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.658920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] PATH_INFO: `/v3/users/89eb02f67c0b4660b056bc172868eec1` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.659063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] PATH_INFO: `/v3/groups` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.659748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.661873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722679f7-2a15-423e-8c31-7395f2dd5cb5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.673413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] RBAC: Authorizing `identity:get_user(user_id=89eb02f67c0b4660b056bc172868eec1)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.674973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4c344-ca6b-42d0-b981-9e8044a4e79e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.677399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 151/308] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/users/89eb02f67c0b4660b056bc172868eec1 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:10.678676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288f5a-3335-4f86-9b2f-28d837f6a352 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.683588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288f5a-3335-4f86-9b2f-28d837f6a352 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.683588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a288f5a-3335-4f86-9b2f-28d837f6a352 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.683588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 152/309] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:10.683588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a288f5a-3335-4f86-9b2f-28d837f6a352 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:10.743029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 158/310] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Sep 1 08:52:10 2022] POST /identity/v3/groups => generated 229 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:52:10.744125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.744409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.744682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.766626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:10.767887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:10.796683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:10.810520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a288f5a-3335-4f86-9b2f-28d837f6a352 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:10.854852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:10.855832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:10.856284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:10.856699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:10.857915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, user.id=89eb02f67c0b4660b056bc172868eec1, scope.system=all) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:10.858376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:10.859771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 159/311] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:10.860603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:10.871151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 153/312] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 08:52:10 2022] GET /identity/v3/role_assignments?user.id=89eb02f67c0b4660b056bc172868eec1&scope.system=all&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:11.144370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:11.235856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:11.256737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7fd84-2991-4042-9ff9-6efb4169ab98 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:11.293187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:11.294077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:11.294519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:11.294946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:11.305501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:11.310699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:11.311847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:11.312865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:11.315179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 160/313] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:11.325901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:11.433555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1388-18f5-4a5a-9cee-73c9fc0c9e18 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:11.470030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:11.470636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:11.470917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:11.471220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:11.471883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:11.472187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:11.474169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:11.487362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 154/314] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:11.488889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:11.602044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368498-f425-40a4-a16e-c12429a7a454 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:11.636324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:11.636947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:11.637226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:11.637498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] PATH_INFO: `/v3/users/ce9c1f057f224e1cb2364a1248ecb32b` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:11.638629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] RBAC: Authorizing `identity:get_user(user_id=ce9c1f057f224e1cb2364a1248ecb32b)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:11.640096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:11.641982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 161/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:11 2022] GET /identity/v3/users/ce9c1f057f224e1cb2364a1248ecb32b => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:11.648461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:11.669629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:11.759317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868470ee-6e33-4c8b-8f54-dbe0b49b430c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:11.794193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:11.794796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:11.795075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:11.795370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:11.863914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:11.865511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:11.887469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:11.969680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 155/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:11.976010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:11.991463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:12.085395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f6854-43c6-4645-8f9e-e8f2ddce43b0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:12.119577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:12.120525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.121035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.121511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.122831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:12.124548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f74cb-5d6f-4a41-a552-f9aaf1cfa461 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:12.126772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 162/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:11 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:12.133577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:12.240714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869e296-ded6-4e1e-98e1-00fe98710b0a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:12.259902 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:12.279797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:12.280784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.281289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.281817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.282738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189, user.id=ce9c1f057f224e1cb2364a1248ecb32b) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:12.283273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:12.284983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d18a-1eee-4dfc-b695-db7df5a45c89 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:12.291650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 156/318] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:12 2022] GET /identity/v3/role_assignments?user.id=ce9c1f057f224e1cb2364a1248ecb32b&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:12.481208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db66d22-da20-444b-b61a-99fefb7fff3c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.481853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db66d22-da20-444b-b61a-99fefb7fff3c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.482341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db66d22-da20-444b-b61a-99fefb7fff3c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.483913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 163/319] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:12.489994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bb823-919a-4c76-bf44-ca4121e29d4c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.490548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bb823-919a-4c76-bf44-ca4121e29d4c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.490548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bb823-919a-4c76-bf44-ca4121e29d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.511940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5f1bb823-919a-4c76-bf44-ca4121e29d4c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:12.513163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5f1bb823-919a-4c76-bf44-ca4121e29d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:12.587661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 157/320] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:12.774779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ad1e-e9f5-499f-899d-642b66ad5990 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.775083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ad1e-e9f5-499f-899d-642b66ad5990 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.775373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ad1e-e9f5-499f-899d-642b66ad5990 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.776871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 164/321] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:12.781759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.782047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.782320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.800950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:12.802791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:12.870921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 158/322] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:12.872906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355f8b-f30f-49ae-bcb1-25cc19b5722f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.873196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355f8b-f30f-49ae-bcb1-25cc19b5722f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.873733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0355f8b-f30f-49ae-bcb1-25cc19b5722f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.874285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:12.875065 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 165/323] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:12.880433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:12.880744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:12.881015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:12.900225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:12.901393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:12.923158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:12.974721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 166/324] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:12.980727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c30f51-b367-421e-815a-718bfbc46863 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.015883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.016462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.016741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.017010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.029080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:13.033934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:13.034853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:13.035746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 159/325] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:13.042863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.054982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.145436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.146390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a809b1d-1073-47b3-a5d9-bd222659d13e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.182204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.182802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.183077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.183374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.184025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:13.184319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.185710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.194527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 167/326] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:13.199838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.250030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55e9e-600a-4f0c-bb87-e7ba9af1b0cf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.251592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.269675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.285503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.286627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.286906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.287338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.288456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.289905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.291500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 160/327] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:13.292846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.302135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e7e86-6bc6-449f-86f3-ad854c8f9500 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.336847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.337475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.337755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.338028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] PATH_INFO: `/v3/users/ad9ecb89c2384414925d36e3f4696c54` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.339184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9ecb89c2384414925d36e3f4696c54)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.340634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.342214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 168/328] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/users/ad9ecb89c2384414925d36e3f4696c54 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:13.342956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.394755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4feb2-6472-4b1d-94ce-b89584cd9a6b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.429493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.430112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.430392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.430664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.440510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:13.443163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ef391-5b37-468f-957e-2eeda632aa05 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:13.445710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:13.446795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 161/329] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:13.447789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.477934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.482438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.483047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.483363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.483636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] PATH_INFO: `/v3/groups` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.484291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.490040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.548007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627b589-5ee8-4fd7-822d-a23d6849d396 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.577240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.582519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.587413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 169/330] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Sep 1 08:52:13 2022] POST /identity/v3/groups => generated 226 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:13.587542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.587542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.587542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.587726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.587793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] RBAC: Adding query filter params (role.id=5fecc05bf35b4b51af54c339e7a42bb7, user.id=ad9ecb89c2384414925d36e3f4696c54, scope.system=all) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:13.587793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.587793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.600047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 162/331] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/role_assignments?user.id=ad9ecb89c2384414925d36e3f4696c54&scope.system=all&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 510 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:13.710221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d6e8-2927-46ad-b726-25c5e83fe6c0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.752158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.752772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.753051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.753323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.753987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:13.754287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.756001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.766309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 170/332] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:13.773285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.867971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:13.881939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2f22-8160-4468-b4e9-fdbd1898e939 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:13.939414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:13.939414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:13.939414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:13.939414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] PATH_INFO: `/v3/users/89eb02f67c0b4660b056bc172868eec1` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:13.939756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] RBAC: Authorizing `identity:get_user(user_id=89eb02f67c0b4660b056bc172868eec1)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:13.939756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b4d9c-5e0f-4b6f-8048-8dbde4b69fac None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:13.939756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 163/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:13 2022] GET /identity/v3/users/89eb02f67c0b4660b056bc172868eec1 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:13.939956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:13.996794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:14.041743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857d67-d703-4ee3-8fe5-03d838f5398b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:14.080276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:14.081587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:14.082025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:14.082511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] PATH_INFO: `/v3/system/users/89eb02f67c0b4660b056bc172868eec1/roles/5d01151b08dd4e3e9c83992195eaa20d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:14.084785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=89eb02f67c0b4660b056bc172868eec1, role_id=5d01151b08dd4e3e9c83992195eaa20d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:14.090934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186822d0-c56c-4bec-8e7b-88c296b96825 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:14.169954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 171/334] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Sep 1 08:52:13 2022] PUT /identity/v3/system/users/89eb02f67c0b4660b056bc172868eec1/roles/5d01151b08dd4e3e9c83992195eaa20d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:15.731270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8be77-62c2-470c-b4d8-bf721f3b5759 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:15.731807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8be77-62c2-470c-b4d8-bf721f3b5759 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:15.731807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8be77-62c2-470c-b4d8-bf721f3b5759 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:15.733369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 164/335] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:15.738751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41cffb-36c0-49c2-a34c-83e19f1c84dc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:15.739233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41cffb-36c0-49c2-a34c-83e19f1c84dc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:15.739438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41cffb-36c0-49c2-a34c-83e19f1c84dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:15.762229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ac41cffb-36c0-49c2-a34c-83e19f1c84dc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:15.763442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ac41cffb-36c0-49c2-a34c-83e19f1c84dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:15.855574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 172/336] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:15.915828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129449-739b-4ee5-86ba-9a33b595fa6e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:15.916164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129449-739b-4ee5-86ba-9a33b595fa6e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:15.916370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129449-739b-4ee5-86ba-9a33b595fa6e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:15.917848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 165/337] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:15.923420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:15.923734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:15.924039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:15.942939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:15.948219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:16.016543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:16.033765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 173/338] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:16.260523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129449-739b-4ee5-86ba-9a33b595fa6e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:16.382415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129449-739b-4ee5-86ba-9a33b595fa6e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:16.424924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:16.425554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:16.425835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:16.426109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:16.439364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:16.445513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:16.446440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:16.447320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 166/339] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:16.452801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:16.587864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128444-5f3f-46ff-b6c5-b23713ecaca9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:16.614971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:16.636959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:16.637579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:16.637854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:16.638127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:16.638797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:16.639095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:16.640578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:16.655532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:16.657236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 174/340] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:16.673508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:16.723341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:16.785802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43b16f-275c-464d-bfa5-1f6d51046990 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:16.831334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:16.831904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:16.832186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:16.832461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:16.840572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d467195-0bd9-4ada-bf9d-d8482d43961d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:16.905072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:16.906086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:16.906581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:16.907051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:16.927363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cc2ff38d-8f01-4471-b45f-f7f887e3aabb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:16.932750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:16.933623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:16.934427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:16.938860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 167/341] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:16.940840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c72d8-5b3e-45bf-8f37-1cf9b0c0cd26 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:16.941339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c72d8-5b3e-45bf-8f37-1cf9b0c0cd26 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:16.941820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c72d8-5b3e-45bf-8f37-1cf9b0c0cd26 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:16.944012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 168/342] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:16.945909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c72d8-5b3e-45bf-8f37-1cf9b0c0cd26 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:16.969196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:16.970975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:17.050720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.076155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.104981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c72d8-5b3e-45bf-8f37-1cf9b0c0cd26 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:17.155574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:17.156535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.157252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.157738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.159025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:17.159695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:17.161832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cf5e3-7399-437c-b165-e963d82b75de None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:17.171881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 169/343] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:17.173725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.174209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.175140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.185946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 175/344] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:52:17.186883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:17.198957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:17.201150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:17.308890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 170/345] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:17.309796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:17.346135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6386c-423e-4e62-b544-36ff5e140e82 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:17.393658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:17.394349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.394628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.394891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.409228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.415659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.457349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c0ae7-d39b-4208-81fb-32d697f40bd0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:17.497266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:17.498722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487137e5-b37c-4854-9f98-75516723b2d2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:17.509149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:17.510081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.510574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.513243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.515582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:17.517342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5a77-25b2-422c-8c8f-421a1e134ffc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:17.518939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 171/346] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:17.524511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473d51-8829-4b21-98de-ee04e90c94ad None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.525408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473d51-8829-4b21-98de-ee04e90c94ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.525888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34473d51-8829-4b21-98de-ee04e90c94ad None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.527702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 172/347] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:17.529526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4976e-c4d3-4779-a207-b9a838829991 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.530127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4976e-c4d3-4779-a207-b9a838829991 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.530606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4976e-c4d3-4779-a207-b9a838829991 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.532374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 173/348] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:17.533373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4976e-c4d3-4779-a207-b9a838829991 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:17.632917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 176/349] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:17.634446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.634446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.634612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.654412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:17.656128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:17.658358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4976e-c4d3-4779-a207-b9a838829991 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:17.708367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:17.709006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.709287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.709556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.714400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.715339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:17.717698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:17.719231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacafb4-a605-4bb4-8089-8f4745a6c682 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:17.723082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 174/350] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:17.724225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.725413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.725413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.743067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:17.744130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 177/351] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:17.744366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:17.744954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:17.830368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 175/352] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:17.831372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:17.894352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c7740-a3c5-4e9d-a739-fa3cc34fdfb9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:17.960643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:17.961188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:17.961459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:17.961723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:17.983603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:17.989692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:17.990648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:17.991778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 178/353] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:17.995150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.028696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9009a-db65-4f78-938f-5bf7803bc48f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.061878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.077743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.078741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.079385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.079882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.081325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.083021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.085312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 176/354] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.088010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.088495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.136615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34d931-b6d5-4c55-b8aa-4fb60bc35384 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.184374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.185390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.185875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.186374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.187331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=3bee042dbad049779321aede9dedec4e) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:18.187844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.192900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.197684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 179/355] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:17 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.198514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.229515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e60b3c-75c7-4078-b9c9-10e337e8af56 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.285062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.285734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.286046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.286322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.288224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:18.288224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.288934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.298522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 177/356] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.300232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.332530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.336769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9ba1-b264-47c6-b932-4b54598ee75f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.380655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.381577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.382143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.382421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.389023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.395936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:18.400877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:18.401925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 180/357] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:18.402906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.442878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872194-0171-4b95-99a3-316d8f360564 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.489199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.489738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.490107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.490471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.492094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.493616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a345e99-9f79-4245-b423-49ae4723995c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.499030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 178/358] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.499030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f953e-9800-4181-baa6-071402c473d2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.499030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f953e-9800-4181-baa6-071402c473d2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.499030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f953e-9800-4181-baa6-071402c473d2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.499030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 179/359] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:18.499513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f953e-9800-4181-baa6-071402c473d2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.535638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0e1a-243f-4517-a45d-dae1dc01e22c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.584802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.585795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.586304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.586765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.601031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:18.606494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:18.607505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 08:52:18.608363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:18.609452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 181/360] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:18.610465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.639598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f953e-9800-4181-baa6-071402c473d2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.641754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.671467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.682813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.683450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.683729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.683999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.748497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262ec07-133d-456b-a2a3-f1e7cebadf10 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.785576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.788005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f18be6-5886-4372-b17e-c0e4303b533d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.791363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.792031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.792304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.792576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.793279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:18.793576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.794923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.805198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 182/361] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.806101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:18.909739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 180/362] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:18.911813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.912269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.912345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.912606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.936689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:18.937359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e325550-292f-460f-a44f-639dab41f7ff None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:18.937767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:18.971859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:18.985402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:18.986021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:18.986970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:18.986970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:18.987352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3bee042dbad049779321aede9dedec4e) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:18.987671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:18.989122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:18.994973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 183/363] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:18.996088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.037331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 181/364] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:19.039215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.129451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec6469-58b8-4456-8858-df7769ec6d1e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.154366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4934c35-5eb7-42e7-ae6b-9febe5f5b192 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.168326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.169261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.169755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.170242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.171196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:19.171712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.173348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.183729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 184/365] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.184987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.198217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.205038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.205450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.205729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.205998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.250918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.301495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6452-20ba-4709-935e-fd29910cfff2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.302788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.308652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.342857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.343915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.344439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.344910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.346269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.347952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.350390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 185/366] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.351471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.427537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 182/367] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:19.428321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.440895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.473097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9c33-6236-44f4-ad65-fe16a5ccbc85 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.507721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.520196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.521851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.522353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.522842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] PATH_INFO: `/v3/users/89eb02f67c0b4660b056bc172868eec1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.524299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] RBAC: Authorizing `identity:get_user(user_id=89eb02f67c0b4660b056bc172868eec1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.526085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.528534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 186/368] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/users/89eb02f67c0b4660b056bc172868eec1 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.529959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.565726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a90604-f056-4620-ba08-1c41b2cd1cf5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.612452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.613089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.613370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.613644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.615003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.617411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.619569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 183/369] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.623213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.672756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf64fdd-3403-43f1-bae0-554a04b846f6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.714781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.715913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.716491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.717009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.718329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.720224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.722385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 187/370] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.723585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.735415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2569fb-44ef-419a-bfd2-55de5ef8f184 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.742340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.763720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:19.772493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.773508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.773795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.774070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.786263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:19.792152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:19.793075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:19.793891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 184/371] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:19.794580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.862694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad176a-10b0-4bc4-9c0d-023c1c1926ae None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.912857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.914263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.914743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.915240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.916804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] RBAC: Adding query filter params (role.id=5d01151b08dd4e3e9c83992195eaa20d, user.id=89eb02f67c0b4660b056bc172868eec1, scope.system=all) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:19.917588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.919685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.930377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba45912-248f-4cef-be08-bffe2fa79c90 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:19.937313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 188/372] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/role_assignments?user.id=89eb02f67c0b4660b056bc172868eec1&scope.system=all&role.id=5d01151b08dd4e3e9c83992195eaa20d => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.938516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:19.976620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:19.977246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:19.977530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:19.977865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:19.978606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:19.978921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:19.980509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:19.990217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 185/373] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:19.991953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:20.043030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.058786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.088610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da13ad-8398-43ff-a741-23d992f3c423 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:20.133331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638fec8-ab11-4f93-be47-714aebe56b40 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:20.140870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:20.141822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.142871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.143367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.145128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:20.147421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:20.149986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 189/374] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:20.155588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:20.178530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:20.179303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.179676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.180065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.180827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:20.181341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:20.182993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:20.193971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 186/375] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:20.199766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:20.284082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4b4c-2f7f-4f0c-9e0b-a201f56906ac None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:20.322516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:20.323457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.323915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.324360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.325282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:20.325774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:20.327527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:20.334180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 190/376] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:20 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:20.347194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f810-e82e-4a31-ac3b-bab0c5176683 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:20.367500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.393513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:20.394050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.394391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.394670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.480839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:20.482320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1dec7-446d-41fd-866f-e107213372ad None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:20.612562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 187/377] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:52:20.618560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:20.627129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.683698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bda5-7721-4105-b339-0f77672bdfab None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.684053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bda5-7721-4105-b339-0f77672bdfab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.684426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bda5-7721-4105-b339-0f77672bdfab None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.686012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 188/378] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:20.691304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.691689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.693081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.730208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:20.731665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:20.747464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.747773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5db8-c9be-4331-9011-9a016b6d043f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:20.789816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13697122-b301-46a1-a201-486cb022b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:20.790468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13697122-b301-46a1-a201-486cb022b17e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:20.790744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13697122-b301-46a1-a201-486cb022b17e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:20.791016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13697122-b301-46a1-a201-486cb022b17e None admin] PATH_INFO: `/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:20.792790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13697122-b301-46a1-a201-486cb022b17e None admin] RBAC: Authorizing `identity:get_project(project_id=0c919d0d1fdc41adb58cac59aa4513ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:20.794240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13697122-b301-46a1-a201-486cb022b17e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:20.796386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 191/379] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:20 2022] GET /identity/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:20.802228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13697122-b301-46a1-a201-486cb022b17e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:20.845218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 189/380] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:20.951374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:20.988091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13697122-b301-46a1-a201-486cb022b17e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:21.038885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:21.039799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.040220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.040631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] PATH_INFO: `/v3/groups/63ae9345a0424971a7d72d60ceb8b541` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.048681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] RBAC: Authorizing `identity:get_group(group_id=63ae9345a0424971a7d72d60ceb8b541)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:21.050323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:21.052637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 192/381] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:20 2022] GET /identity/v3/groups/63ae9345a0424971a7d72d60ceb8b541 => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:21.058095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:21.190472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d915-a2cf-47fc-a4b5-165faeab20b3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:21.237196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:21.237728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.238005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.238274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.239534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] RBAC: Adding query filter params (group.id=63ae9345a0424971a7d72d60ceb8b541, role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=0c919d0d1fdc41adb58cac59aa4513ea) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:21.239989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:21.241810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:21.247938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 190/382] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:21 2022] GET /identity/v3/role_assignments?group.id=63ae9345a0424971a7d72d60ceb8b541&scope.project.id=0c919d0d1fdc41adb58cac59aa4513ea&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:21.301206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:21.440884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908a07d-4a9e-43d5-ab1b-37749f388b48 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:21.476737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:21.477384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.477638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.477901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.479847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:21.497741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:21.498692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:21.499574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 193/383] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:21.503272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:21.516817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:21.649585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906beed-f63d-49e6-8ae4-f37b7e7099ce None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:21.693299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:21.693918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.694196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.694465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.695150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:21.695456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:21.696848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8942988-5ed0-49b3-9bfb-cab4a0559faa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:21.705955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 191/384] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:21.712190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:21.867617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f8ecd-08a3-4a75-baaf-b95658361c59 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:21.872010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574978a-c604-4e56-aeb3-b5b43a6428e0 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.872288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574978a-c604-4e56-aeb3-b5b43a6428e0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.872557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574978a-c604-4e56-aeb3-b5b43a6428e0 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.873884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 192/385] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:21.889230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.889497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.889764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.916110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:21.919058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:21.934403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:21.935067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:21.935367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:21.935638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:21.937683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:21.939198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:21.940761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 194/386] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:21 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:21.947923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:22.038214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:22.044287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 193/387] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 08:52:22.057695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:22.126819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ac115-3ce5-4881-a4c9-fbdc2f3fa0be None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:22.177478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:22.178126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.178401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.178665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.179841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:22.181276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:22.182902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 195/388] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:21 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:22.188576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:22.329924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44377a8-2a13-4f86-926a-85078a340d9e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:22.373644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:22.374314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.374590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.374853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.376776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, group_id=199e48f3e8f7450ca5dff785a1f3948e, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:22.378284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:22.409274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:22.466279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 194/389] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 08:52:22 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:22.536505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279918e-5324-415d-8f0e-cdcbe99efcaa None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:22.570489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:22.579878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:22.580530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.580808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.581078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.593059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-da31756f-1305-47dd-86c7-1d19eb02d385 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:22.602456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:22.603463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:22.604333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:22.607425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 196/390] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:22.612227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:22.719745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:22.765997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2d86-0abe-4651-86f7-4ebc9eb580cc None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.766623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2d86-0abe-4651-86f7-4ebc9eb580cc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.767150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2d86-0abe-4651-86f7-4ebc9eb580cc None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.769947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0535870-3966-4d3a-bca8-6b8283e4386c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:22.770207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 197/391] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:22.776520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ea15b-ba13-4edc-8a50-9ab183ceeb46 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.777095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ea15b-ba13-4edc-8a50-9ab183ceeb46 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.777585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ea15b-ba13-4edc-8a50-9ab183ceeb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.805286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-892ea15b-ba13-4edc-8a50-9ab183ceeb46 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:22.806819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-892ea15b-ba13-4edc-8a50-9ab183ceeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:22.817035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:22.817287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.817593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.817957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.818634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:22.818937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:22.820901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:22.833624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 195/392] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:22.839038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:22.899259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 198/393] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:22.901316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80de02-d8df-4306-8e05-add2d3a248fc None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.901944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80de02-d8df-4306-8e05-add2d3a248fc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.902439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80de02-d8df-4306-8e05-add2d3a248fc None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.904104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 199/394] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:22.909950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.910560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.911084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.940739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:22.942889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:22.949861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a82b0-566f-4ef6-a540-3116fbed3595 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:22.975549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:22.993014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:22.993652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:22.993933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:22.994209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:22.995964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:22.997419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:23.008500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 196/395] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:22 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:23.009292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.080753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 200/396] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:23.144555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10277651-4550-44f2-b79f-a338b2eb1fbe None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.194743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.195458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.195740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.196016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.197167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:23.198616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:23.200569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 197/397] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:23.206269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.258758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.302167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:23.360101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d58e93-adca-4cac-b6fd-0cd02213ea61 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.384583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:23.416782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.417456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.417735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.418006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.422633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bee042dbad049779321aede9dedec4e, group_id=199e48f3e8f7450ca5dff785a1f3948e, role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:23.424112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:23.451916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9756042-aff7-4d4c-8081-d9d4b2b8af29 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.488740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.489997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.490287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.490561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.503881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 201/398] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 08:52:23 2022] PUT /identity/v3/projects/3bee042dbad049779321aede9dedec4e/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:23.505398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:23.505649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:23.514133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:23.516206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:23.517055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:23.519174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 198/399] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:23.528369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.661880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682289d9-5792-4b42-bcc1-42e339ff2aa6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.675868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:23.709356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.710462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.710949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.711562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.740340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:23.747845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:23.748992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:23.750013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:23.751532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 202/400] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:23.758255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.770058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab36033-77df-48b0-bc12-1ccabfd7ee59 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.795927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:23.821040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.821886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.822278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.822571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.823131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:23.823432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:23.824809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedf3b2-2cc2-4e24-a30f-633d24079c4f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:23.834047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 199/401] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:23.835089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486f30-a786-4784-8293-7f5f40856d30 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.835433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486f30-a786-4784-8293-7f5f40856d30 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.835710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7486f30-a786-4784-8293-7f5f40856d30 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.837030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 200/402] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:23.841576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486f30-a786-4784-8293-7f5f40856d30 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:23.902652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa059ae-3cef-4e79-b422-a6ac634cb7ff None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.948412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:23.949520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.950294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.951007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.951872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:23.952515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:23.953659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7486f30-a786-4784-8293-7f5f40856d30 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:23.954274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf0dac-ddbc-42df-a62e-56f64ec35fb1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:23.965628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 203/403] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:23.967210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:23.967656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:23.968089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:23.991532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:24.000376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:24.000491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.000491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.000491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.000491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.000771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.001310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.003295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 201/404] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:23 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:24.004233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.027319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:24.056631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:24.080240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 204/405] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:24.086123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.125110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5524e06-5257-49c3-8f5a-ad62ac229fba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.172183 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.172822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.173101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.173370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.174767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.176354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.178832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 202/406] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:24 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:24.184734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.300172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b42c3-5d58-4eea-8a1f-6bcf0133a02f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.315492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:24.321753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46427b-f754-4c15-a599-5a9e6aba6383 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.337171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.337810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.338095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.338373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.339574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.341142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.343235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 203/407] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:24 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:24.348316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.399711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.401816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.402097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.403350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.405286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.406730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.408397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 205/408] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:24 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:24.414335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.470013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:24.530406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107ccdd-e1da-4a6d-b97c-af7d985c10ed None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.571128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3351-ae96-4fd5-974f-b477e686bd9a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.573575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.574237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.574518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.574796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.577382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, group_id=199e48f3e8f7450ca5dff785a1f3948e, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.579315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.605361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.606012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.606284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.606547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.608811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] RBAC: Authorizing `identity:create_grant(project_id=04d4e0ea1c424baa9feec5f4aef22189, group_id=199e48f3e8f7450ca5dff785a1f3948e, role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:24.610363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78c13-7ab0-4ba0-8527-b878f06316da None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:24.681010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 204/409] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 08:52:24 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:52:24.682578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.727670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 206/410] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 08:52:24 2022] PUT /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189/groups/199e48f3e8f7450ca5dff785a1f3948e/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:24.734073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407da3-f9ce-4f37-82f7-92bb334a23a4 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.734418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407da3-f9ce-4f37-82f7-92bb334a23a4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.734680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e407da3-f9ce-4f37-82f7-92bb334a23a4 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.736268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 207/411] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:52:24.743182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.743829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.744317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.750318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:24.764844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:24.766401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:24.839129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af901d-780e-4241-a326-d8325ddbf178 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:24.903184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 208/412] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:24.918837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:24.919894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:24.920371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:24.920837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:24.935754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:24.941701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:24.943406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:24.944230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:24.946212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 205/413] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:24.952411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:24.977096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:25.057668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730a08-ca44-4f5d-8fb3-06eb5a7c1b53 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.101722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.103070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.103571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.104032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.104975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:25.105784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:25.107385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:25.117731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 209/414] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:25.123127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.223517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.291369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ecbe-652f-4aea-be7e-7fcba2f62081 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.293185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:25.327543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:25.346236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab992e6-f970-40e8-a9e1-52b281ec549f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.363688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.365188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.365494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.365765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] PATH_INFO: `/v3/groups/63ae9345a0424971a7d72d60ceb8b541` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.367827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] RBAC: Authorizing `identity:get_group(group_id=63ae9345a0424971a7d72d60ceb8b541)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:25.370004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:25.373614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 206/415] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:25 2022] GET /identity/v3/groups/63ae9345a0424971a7d72d60ceb8b541 => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:25.380766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.405863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.406830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.408621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.409096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.436062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:25.441463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:25.443085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:25.443921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:25.451834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 210/416] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:25.457085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.519005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef04ec-f961-4bb2-8f07-05f9c49041d9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.557055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.557596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.562725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.562725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] PATH_INFO: `/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.562725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] RBAC: Authorizing `identity:get_project(project_id=0c919d0d1fdc41adb58cac59aa4513ea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:25.562725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:25.562725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 207/417] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:25 2022] GET /identity/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:25.572810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.615354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:25.677691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41284e4-df98-4fae-a613-6c81e5fcbf48 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.681403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be95f56-3504-439d-9ded-e9e08cbd9d83 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.726042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.727090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.727617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.728135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.728987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:25.729491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:25.731049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:25.740896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 211/418] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:25.742471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.743468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.743948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.744454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] PATH_INFO: `/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea/groups/63ae9345a0424971a7d72d60ceb8b541/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.746391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:25.750585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] RBAC: Authorizing `identity:create_grant(project_id=0c919d0d1fdc41adb58cac59aa4513ea, group_id=63ae9345a0424971a7d72d60ceb8b541, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:25.752004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:25.752112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fab437-428f-4c98-b999-86f150a3ddfb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:25.819958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 208/419] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 08:52:25 2022] PUT /identity/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea/groups/63ae9345a0424971a7d72d60ceb8b541/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:25.822118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d98c5-0827-4ca2-8f7a-2e219a4d2058 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.822623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d98c5-0827-4ca2-8f7a-2e219a4d2058 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.822909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d98c5-0827-4ca2-8f7a-2e219a4d2058 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.824607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 209/420] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:25.833230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.833230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.833230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:25.855093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:25.856769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:25.896401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036e54-5f69-4ab7-80a6-559db2b970c9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:25.951491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 210/421] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:25.968809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:25.969435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:25.969714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:25.969984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.052314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:26.053772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:26.056679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:26.187417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 212/422] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:26.193411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.240579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:26.271833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.344181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd9648-3a6f-4d9d-bb34-02ab8f789828 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.394825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.395520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.395794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.396062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.397164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:26.398627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:26.400255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 211/423] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:26.416061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.435139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aac29-6f58-43d5-9e8c-535673eb1ed8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.443565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:26.473263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.474452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.474732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.475001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.485913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:26.491051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:26.492229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:26.493042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 213/424] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:26.504285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.569951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:26.580877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894159-3c80-4a4f-aaa4-05a5d3313807 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.629509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fe03-1dad-4aa2-ad37-c4ba31ba4377 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.631591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.637221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.637221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.637221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.637221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:26.637221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:26.637661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 212/425] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:26.642926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.700494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.701015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.701289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.701555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.702210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:26.702503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:26.703890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:26.721945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 214/426] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:26.726182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:26.779541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:26.826754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1401b-1a31-4e2c-b241-a86cf714976f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.865952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.866803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.867077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.867388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:26.868196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=3bee042dbad049779321aede9dedec4e) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:26.868505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:26.869932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103632eb-b90a-4448-8049-87b10f1f0b24 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:26.876495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 213/427] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:26.919299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bdc73-cd59-4a21-84a2-e3672926147e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:26.981088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:26.981625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:26.981901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:26.982168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.080436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:27.081873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:27.102869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0128-d456-40b2-9642-2e9cc6542384 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.103224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0128-d456-40b2-9642-2e9cc6542384 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.103450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0128-d456-40b2-9642-2e9cc6542384 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.105048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 214/428] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:27.113888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.121085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62132e-5124-43e9-90fb-0e8b12f434a8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.121365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62132e-5124-43e9-90fb-0e8b12f434a8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.121650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62132e-5124-43e9-90fb-0e8b12f434a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.142794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7f62132e-5124-43e9-90fb-0e8b12f434a8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:27.144013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7f62132e-5124-43e9-90fb-0e8b12f434a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:27.176684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.216581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 215/429] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:27.218584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.243208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 215/430] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 08:52:27.329162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9e62-33c4-449f-9187-18139120000c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.338865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24662-6251-4baf-b343-4d0f15a6634e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.339171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24662-6251-4baf-b343-4d0f15a6634e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.339431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a24662-6251-4baf-b343-4d0f15a6634e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.340768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 216/431] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:27.345739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.346015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.346286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.365593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.365771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:27.367806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:27.378407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.379071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.379379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.379741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] PATH_INFO: `/v3/projects/3bee042dbad049779321aede9dedec4e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.381206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] RBAC: Authorizing `identity:get_project(project_id=3bee042dbad049779321aede9dedec4e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:27.383323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:27.385876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 216/432] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/projects/3bee042dbad049779321aede9dedec4e => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:27.391449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.466532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 217/433] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:27.517873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f8dd-09d5-4f13-adbc-3e7a8fb5a01f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.554273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.554273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.554273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.554612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.555500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:27.557331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:27.558761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 217/434] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:27.570119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.570626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.652614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.700895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.704975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf966f9c-22b3-483b-98f9-68f60321aa34 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.717464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f11515-40af-46a2-9918-633e4630aa0d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.753940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.754577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.755474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.756492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.764527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.765527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.766130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.766623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.768569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3bee042dbad049779321aede9dedec4e) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:27.769008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:27.769584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:27.772584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:27.776505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:27.777457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:27.778306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 218/435] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:27.779082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.783141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 218/436] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=3bee042dbad049779321aede9dedec4e&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:27.784646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.923758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a34ea-1484-4b62-a61a-f61ce7df2b24 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.929424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc465f1-cd0d-42d6-b9fb-f4f0f1573eb5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:27.956026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:27.973190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.974610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:27.974764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.975048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.975335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:27.975450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.975610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:27.975871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:27.976035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:27.976346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:27.977696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:27.986515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 219/437] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:27.991147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:27.994051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:27.996290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:27.997207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:27.998023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:28.001080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 219/438] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:28.003621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.017302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.119633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ea13-d0b8-4358-8972-317459e7ac0d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.133974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2a7f-8767-4838-b790-975c8427700a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.159915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.160529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.160811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.161675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.182407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.183026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.183338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.183609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.184301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:28.184606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.187867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c779ce-8479-4d2e-b4a5-b4e9ee34cd7a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.198655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 220/439] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:28.199833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60946c5-1628-409a-8463-29eb95c94121 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.200117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60946c5-1628-409a-8463-29eb95c94121 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.200387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60946c5-1628-409a-8463-29eb95c94121 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.201710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 221/440] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:28.206115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60946c5-1628-409a-8463-29eb95c94121 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.247945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.249773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f922031-9a13-46f5-bdad-1957900ece7c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.264652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.265465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.319291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60946c5-1628-409a-8463-29eb95c94121 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.347722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 220/441] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:28.349564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.349853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.350124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.351964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.352556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.352994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.353103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.368255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:28.369446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:28.420095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.421530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.434609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 221/442] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:28.435459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.480187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.481261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.536844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b33cc-c4f9-479b-aa76-ee6ff8096404 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.542872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 222/443] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:28.543786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.574427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.575062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.575371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.575646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.576759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.578202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.579868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 222/444] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:28.585389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.660447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540557f-0f7f-4080-a6ea-b8d0bd46b432 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.691150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe62dd-2235-4212-969b-d0b150a64923 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.696859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.697575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.697846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.698113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] PATH_INFO: `/v3/projects/04d4e0ea1c424baa9feec5f4aef22189` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.699257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] RBAC: Authorizing `identity:get_project(project_id=04d4e0ea1c424baa9feec5f4aef22189)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.700673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.702277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 223/445] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/projects/04d4e0ea1c424baa9feec5f4aef22189 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:28.703025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.705376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.719615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.727831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.728453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.728731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.729087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.730188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.731676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.733226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 223/446] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:28.733975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.804499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bb92-9162-418c-81a3-93a6f2b7e389 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.832243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a338675-fee3-4eed-9d47-d0fe1639ddec None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.836963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.837575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.837848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.838117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.847872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:28.853348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:28.854447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:28.855332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 224/447] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:28.856063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.870347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.870973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.871287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.871568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] PATH_INFO: `/v3/groups/199e48f3e8f7450ca5dff785a1f3948e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.872676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] RBAC: Authorizing `identity:get_group(group_id=199e48f3e8f7450ca5dff785a1f3948e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.874134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:28.875716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 224/448] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/groups/199e48f3e8f7450ca5dff785a1f3948e => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:28.876496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:28.926195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.927732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:28.956182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e709b-1588-4c3f-ad84-53020abed106 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.977347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a8e8-820e-4d80-a711-8be014de220b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:28.991718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:28.992687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:28.993118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:28.993396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:28.994121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=5fecc05bf35b4b51af54c339e7a42bb7, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:28.994433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:28.995879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.001513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 225/449] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=5fecc05bf35b4b51af54c339e7a42bb7 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:29.002286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:29.014835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.015551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.015828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.016096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.016778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:29.017079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.018498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.028745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 225/450] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:29.034760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:29.121684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a2dfa-8545-4614-bfa9-c058b266d1ab None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:29.148363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c072b-8e68-4f0f-ab6c-ad9063fadb4c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:29.159827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:29.162839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.163828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.164130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.164396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.165405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] RBAC: Adding query filter params (group.id=199e48f3e8f7450ca5dff785a1f3948e, role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=04d4e0ea1c424baa9feec5f4aef22189) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:29.165704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.167659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.171185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:29.174861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 226/451] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:29 2022] GET /identity/v3/role_assignments?group.id=199e48f3e8f7450ca5dff785a1f3948e&scope.project.id=04d4e0ea1c424baa9feec5f4aef22189&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:29.186278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.186942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.187325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.187815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.273110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.274605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.383869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 226/452] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 08:52:29.389532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:29.492661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7006bd-64d2-46fd-8afb-e6b7ea6a349c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:29.527303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.527871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.528126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.528392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] PATH_INFO: `/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.529495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] RBAC: Authorizing `identity:get_project(project_id=0c919d0d1fdc41adb58cac59aa4513ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.530916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.532619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 227/453] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:52:29 2022] GET /identity/v3/projects/0c919d0d1fdc41adb58cac59aa4513ea => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:29.538716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:29.550073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:29.634303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e390cc-cd8a-4d51-840d-06d1b2ab54c9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:29.667491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.668102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.668376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.668645 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] PATH_INFO: `/v3/groups/63ae9345a0424971a7d72d60ceb8b541` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.669721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] RBAC: Authorizing `identity:get_group(group_id=63ae9345a0424971a7d72d60ceb8b541)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.671168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb106dd6-ec8b-47f0-bd35-e8fca9217080 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.672707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 227/454] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 08:52:29 2022] GET /identity/v3/groups/63ae9345a0424971a7d72d60ceb8b541 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:29.679049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:29.743076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:29.774531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67f5e-74a5-420e-9e2e-9e9cddf19a68 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:29.806742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:29.807296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:29.807571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:29.807834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:29.808535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] RBAC: Adding query filter params (group.id=63ae9345a0424971a7d72d60ceb8b541, role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=0c919d0d1fdc41adb58cac59aa4513ea) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:29.808830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:29.810230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220cc0-0e3d-4e93-9da1-90b5661a5abc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:29.815694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 228/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 08:52:29 2022] GET /identity/v3/role_assignments?group.id=63ae9345a0424971a7d72d60ceb8b541&scope.project.id=0c919d0d1fdc41adb58cac59aa4513ea&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:32.325779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b572e9-664e-40fe-9a92-a0735469fb53 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.326799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b572e9-664e-40fe-9a92-a0735469fb53 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.327350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b572e9-664e-40fe-9a92-a0735469fb53 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.329128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 228/456] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:32.335565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211a2bb-f416-45c4-a8c7-5972f06472ec None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.336041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211a2bb-f416-45c4-a8c7-5972f06472ec None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.336118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211a2bb-f416-45c4-a8c7-5972f06472ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.364723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7211a2bb-f416-45c4-a8c7-5972f06472ec None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:32.366047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7211a2bb-f416-45c4-a8c7-5972f06472ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:32.436681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbff13-eccb-420d-89f5-9449d24181e7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.437002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbff13-eccb-420d-89f5-9449d24181e7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.437222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbff13-eccb-420d-89f5-9449d24181e7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.438566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 229/457] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:32.444561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221712c0-b9a0-4c09-ba9f-f1416d2d9e82 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.445106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221712c0-b9a0-4c09-ba9f-f1416d2d9e82 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.445594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221712c0-b9a0-4c09-ba9f-f1416d2d9e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.453972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 229/458] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:32.481597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-221712c0-b9a0-4c09-ba9f-f1416d2d9e82 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:32.483180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-221712c0-b9a0-4c09-ba9f-f1416d2d9e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:32.598052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 230/459] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:32.664052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c466-ed4b-492b-8d45-b1c3bfdac953 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.664512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c466-ed4b-492b-8d45-b1c3bfdac953 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.664881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c466-ed4b-492b-8d45-b1c3bfdac953 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.666319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 230/460] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:32.671663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f768a-12fa-4e6e-b595-6f4d73404d2f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.672162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f768a-12fa-4e6e-b595-6f4d73404d2f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.672608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f768a-12fa-4e6e-b595-6f4d73404d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.696632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e59f768a-12fa-4e6e-b595-6f4d73404d2f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:32.698174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e59f768a-12fa-4e6e-b595-6f4d73404d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:32.700723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:32.789675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 231/461] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:32.829904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484f25e-152c-4fd5-9aba-c5591ad1b990 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.830530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484f25e-152c-4fd5-9aba-c5591ad1b990 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.831002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484f25e-152c-4fd5-9aba-c5591ad1b990 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.832841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 231/462] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:32.838088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:32.838372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:32.838640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:32.857208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:32.858294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:32.926559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484f25e-152c-4fd5-9aba-c5591ad1b990 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:32.938961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 232/463] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:32.971583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.019599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.063478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.096907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484f25e-152c-4fd5-9aba-c5591ad1b990 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.118414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cbe11-edac-472e-9a44-b6e4020a91ab None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.155808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.156498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.156778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.157047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.169021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.170629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:33.187398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:33.188342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:33.189201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:33.189201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.189201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:33.189201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 233/464] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:33.197415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.201145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.201781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.202060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.202335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.213727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.215276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:33.230556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:33.231515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:33.232453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:33.232453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.232453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:33.232453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 232/465] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:33.233076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.314667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32869-b9c1-4e26-928a-e584f3d04524 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.337174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.339538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f1ab3-0938-43b0-a932-4514551f647e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.352633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.353259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.356345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.356470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.356550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.365361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.367073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.374737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.375308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.375597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.375871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.376515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:33.376816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.378177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:33.383257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:33.384167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:33.385110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:33.385110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.385110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:33.385110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 234/466] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:33.385765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.401881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 233/467] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:33.403094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.495457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78480d6-69cd-4a81-9d1c-08e8162ad444 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.507659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0e68-5e34-4a52-8c7a-728e4fc3962d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.531019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.531721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.531997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.532264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.532897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:33.533193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.534569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.543530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.544161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.544436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.544710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.552295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 235/468] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:33.552295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.565665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.567199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:33.582310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:33.583560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:33.584731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:33.584731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:33.584731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:33.588374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 234/469] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:33.588501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.588590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.616149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.665169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf97054-c8fb-4c29-b256-5a30cebb440b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.694769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abb6ad-9d42-4387-8443-c29c411ff05b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.704623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.705413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.705852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.706270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.707080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:33.707406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.709249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.723148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 236/470] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:33.723148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.737686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.738286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.738561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.738825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.740281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.741965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7eabb5-6588-4834-9611-d5a3dc74dc69 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.818544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:33.826919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ebab-5a4e-41bf-aad6-f86f2c7edb7e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:33.855069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 235/471] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:33 2022] POST /identity/v3/users => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:33.861436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:33.862053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.862333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.862606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.863316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:33.865738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aabd8-4a35-46c0-9135-545e5b832160 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:33.865898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:33.866086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aabd8-4a35-46c0-9135-545e5b832160 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:33.866294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aabd8-4a35-46c0-9135-545e5b832160 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:33.868019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 236/472] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:33.868808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aabd8-4a35-46c0-9135-545e5b832160 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:33.944603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 237/473] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:33 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:33.944603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:34.005043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:34.038091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aabd8-4a35-46c0-9135-545e5b832160 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:34.093783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:34.094464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.094745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.095006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.095803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:34.096114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:34.097485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5e34a-af68-4583-935f-e713e6ab2e36 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:34.100351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bb811-6694-4a3c-9af1-55c6ca9cbcfe None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:34.111935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 237/474] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:34.113189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.113463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.113720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.133537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:34.134730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:34.149663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:34.150387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.150663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.151045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.151728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:34.153540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:34.209081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 238/475] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:34.209838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:34.254329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 238/476] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:33 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:34.303147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:34.347088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13afff-27ec-4893-8eef-e31532c9c0d1 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:34.390382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:34.391013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.391320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.391592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.392247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:34.393698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:34.488065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:34.493210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:34.500981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 239/477] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:34 2022] POST /identity/v3/users => generated 249 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:34.621988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2aa3c-cab6-43aa-ba16-99c8304ae609 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:34.660552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:34.661700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.661972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.662235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.662907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:34.664806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:34.703338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.sql.core [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 08:52:34.703338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 08:52:34.703338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [parameters: {'id': '86990665b2744f6eb4d2263cef987146', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 08:52:34.703338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81634) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Sep 01 08:52:34.720720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:52:34.721692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application yield resource Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise self.value Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application session.commit() Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 08:52:34.722647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.flush() Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application compat.raise_( Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 08:52:34.724328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:52:34.725563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:52:34.726442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [parameters: {'id': '86990665b2744f6eb4d2263cef987146', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:34.728512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 08:52:34.732913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:34.734950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 239/478] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 08:52:34 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:34.735086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:34.847642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecac67-6632-4d3d-ba83-e30880f75143 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:34.876173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:34.884647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:34.885275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:34.885599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:34.888639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:34.896971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-5fe1d90f-2fd5-48e4-8552-f10d5efc52e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:34.902063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:34.904249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:34.906910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 240/479] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:52:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:34.923051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:34.986109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:35.024678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c97b25-9e6c-4333-a780-0ff041513343 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:35.068826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:35.069712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:35.070158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:35.070567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:35.071403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:35.071850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:35.073436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626dfc-6c9b-48a6-b840-19e9a4ea3838 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:35.082710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 240/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:36.088481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ff90-ec50-45bd-8a7a-a2f145cdc06d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.089517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ff90-ec50-45bd-8a7a-a2f145cdc06d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.090029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ff90-ec50-45bd-8a7a-a2f145cdc06d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.091836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 241/481] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:36.097292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34e5c-a6f4-4dc3-8aff-55cde631d4c1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.098069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34e5c-a6f4-4dc3-8aff-55cde631d4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.099262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34e5c-a6f4-4dc3-8aff-55cde631d4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.119750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9ac34e5c-a6f4-4dc3-8aff-55cde631d4c1 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:36.121231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9ac34e5c-a6f4-4dc3-8aff-55cde631d4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:36.194163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 241/482] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:36.298918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423b365-6ac3-4864-919a-f2ce219b24e8 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.299549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423b365-6ac3-4864-919a-f2ce219b24e8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.300013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423b365-6ac3-4864-919a-f2ce219b24e8 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.302306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 242/483] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:36.307299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.307797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.308259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.331563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:36.333068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:36.395954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:36.410182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 242/484] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:36.547997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423b365-6ac3-4864-919a-f2ce219b24e8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:36.692975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423b365-6ac3-4864-919a-f2ce219b24e8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:36.721904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:36.756719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:36.757346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.757628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.758063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.781023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:36.788801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:36.789690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:36.790498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:36.807923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 243/485] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:36.816868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:36.903510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:36.911522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190925d-cb19-478a-84d8-ae6390c2c564 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:36.955419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fa62-903e-445e-803d-5316c68ec3a9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:36.995405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:36.996266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.996638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.997016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:36.997222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:36.997879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:36.998163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:36.998339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:36.998747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:36.999191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.000146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417be5-27f7-4010-a069-afc40052f963 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.011242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:37.011587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 244/486] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-017b45a5-8013-4dd8-a415-86610b431faa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:37.016879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:37.017753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:37.018577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:37.019262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5e4f-1d5d-4a93-ab4b-d226d3e899a7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.019262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5e4f-1d5d-4a93-ab4b-d226d3e899a7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.019262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5e4f-1d5d-4a93-ab4b-d226d3e899a7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.019262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 245/487] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:37.023851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f5e4f-1d5d-4a93-ab4b-d226d3e899a7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.025385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 243/488] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:37.026890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.027388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.027852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.092989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:37.094663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:37.145307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.170141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 244/489] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:37.176293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.221890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f5e4f-1d5d-4a93-ab4b-d226d3e899a7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.237242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.263385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.264446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.264993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.265620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.282818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.284693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-b8ec3227-3756-4ab1-9814-01174311e0ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:37.300230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:37.301180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:37.302768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:37.302768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:37.302768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:37.304878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 246/490] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:37.306359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3284-7b2e-4235-8af9-6435c63e070f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.306833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3284-7b2e-4235-8af9-6435c63e070f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.307353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3284-7b2e-4235-8af9-6435c63e070f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.309044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 247/491] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:37.309987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3284-7b2e-4235-8af9-6435c63e070f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.314090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece9aba-4eda-4df0-9187-618001f43ee4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.353131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.353745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.354014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.354288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.354982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:37.355436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.356821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18a502-374e-4c5e-a297-588d21bf635f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.372395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 245/492] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:37.372395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.372395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.372395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.392037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:37.393345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:37.436376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.438080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3284-7b2e-4235-8af9-6435c63e070f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.471979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 246/493] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:37.473142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.495260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.496179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.496627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.497118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.497963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:37.498441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.500002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.514587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 248/494] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:37.515926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.548065 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.602098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a30ab-7c19-4227-801a-162ec810e5f4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.639471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.639855 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9a57-2f8c-4d5b-b8e2-89757007f9c0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.640098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.640371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.640637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.656832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.658772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:37.674295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:37.675273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:37.676207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:37.676207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:37.676207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:37.676207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 247/495] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:37.677019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.679445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.680224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.680345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.680596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.691787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d1f30bc3-97d1-4ebd-b8c8-0ed06fbac9fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:37.697562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:37.698475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:37.699692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 249/496] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:37.700377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb306aea-12b4-4945-a92c-a5fb4d5beb92 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.700377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb306aea-12b4-4945-a92c-a5fb4d5beb92 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.700377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb306aea-12b4-4945-a92c-a5fb4d5beb92 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.701709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 250/497] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:37.706164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb306aea-12b4-4945-a92c-a5fb4d5beb92 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.712027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.788908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba978f4-b3ed-4ebb-81fc-e2fc0cbe8162 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.789576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.809072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb306aea-12b4-4945-a92c-a5fb4d5beb92 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.823959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.824582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.824861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.825128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.832531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.834435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.842589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.843210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.843489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.843757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.844387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:37.844685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.846026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea9243-8162-4603-af54-de467d423a11 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:37.846707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:37.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 248/498] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:37.848830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.866924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 251/499] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:37.866924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.866924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.866924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.882094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:37.883310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:37.931915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:37.947468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9964ad-7164-4f34-9957-ae49e18e204d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:37.949088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 252/500] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:37.949925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:37.984037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:37.984800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:37.985404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:37.985689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:37.986621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:37.986925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:37.988818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:37.998075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 249/501] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:37.998075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.036546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.068961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b86f8-89b9-46f2-aa03-0154bd15b48c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.102490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406cda-43de-4c36-9048-6e26b35f0f13 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.107495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.108364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.108820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.109271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.121256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:38.126782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:38.127721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:38.128541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:38.130740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 253/502] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:38.131846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.140313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.140915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.141191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.141458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.142176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:38.142476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.143962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.181407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 250/503] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:52:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:38.182232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.189103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.240221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c8a2a-42c3-4d9f-a735-165d472d78f5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.268997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.274590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.275240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.275517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.275786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.283166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.284598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.286835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f500-636d-4a00-9c1d-2f0b0c076151 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:38.297026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:38.298152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 254/504] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:38.299393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.326075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.326625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.326911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.327227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.337598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.339065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:38.353525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:38.354624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:38.355528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:38.355528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.355528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:38.355528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 251/505] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:38.355809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.402253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc29b65-06c3-47d9-934d-6aa7b67d337e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.411826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.435289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.435884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.436160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.436428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.437074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:38.437372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.438734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.447417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 255/506] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:38.451997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.460228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34ae0e-4868-43d0-b0a3-1535326c34b2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.493555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.494156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.494431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.494697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.501917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.547481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0372-4796-4c18-8593-3b51e110bbbc None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.564052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.565522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.582380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.582961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.583518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.583816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.594853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.596325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:38.610461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:38.611426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:38.612287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:38.612287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.612287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:38.612287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 256/507] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:38.612502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.622476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.670464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 252/508] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:38.671661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.714230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fd9cf-34a0-445a-8983-96373f1227e8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.721360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.747892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.748489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.748763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.749028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.750052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:38.750052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.751590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.774884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c6b4-80f7-4fd3-babf-e63335757ee7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.789502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 257/509] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:38.790319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.809060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.809659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.809942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.810215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.810856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:38.811193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.812570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.824304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 253/510] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:38.825134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.842478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.893461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4f472-3922-4862-8781-d1b24f3e3124 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.925407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12de13-200f-44e6-b4ae-17293b07e46b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:38.927796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.928420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.928695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.928960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.938879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.940374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:38.954694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:38.955778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:38.956675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:38.956675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:38.956675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:38.956675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 258/511] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:38.956901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:38.960536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:38.961138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:38.961474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:38.961784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:38.962413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:38.962716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:38.964142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:38.976356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 254/512] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:38.977141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:38.977258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.056384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.060100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404880-9c68-4369-8c97-e5761e6240c2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.078092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5b2f1-1488-4831-ab3c-21dce4cf7319 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.092415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.093137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.093323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.093583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.104406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.105854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.111462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.112050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.112324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.112589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:39.121134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:39.122183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:39.123076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:39.123076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:39.123076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:39.123076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 259/513] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:39.123076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.182620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.184095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.189336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.222565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70338aa-38a4-4512-9d2e-35fee90cefd1 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.257535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.258131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.258402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.258665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.259131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.265994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.267458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:39.278785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:39.279788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 260/514] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:39.280749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.282049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 255/515] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:39.282408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.381379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335052dd-2558-4a93-bcf2-66874c56a6eb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.382189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31f94-df6f-4d5d-8895-b7089763daa1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.408419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.416807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.416968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.417403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.417564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.417677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.417845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.417997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.418119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.418629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:39.418753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.418923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.420230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.420322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.432136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 261/516] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:39.432937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.517034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 256/517] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:39 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:39.518144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.530341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1b2e-638c-4986-9950-ae024be9783a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.577757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.579347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.579975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.580250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.580584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.581240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:39.581595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.583908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.597015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 262/518] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:39.597899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.656432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.661217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bc6d5-f723-4bd6-9edc-c0476b7b38f1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.706035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.707025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.707477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.708639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.715331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4298-8e18-4480-9148-195508f7ece0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.723210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.725720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:39.740708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:39.741932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:39.742821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:39.742821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:39.742821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:39.742821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 257/519] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:39.743038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.758349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.759330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.759627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.759900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.760954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:39.761414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.763143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.787037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 263/520] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:39.791827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.837151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.852782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70e91-1d9e-4863-9398-e506a9b6545d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.887439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.888065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.888342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.888612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.897218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.898895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.899050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b4dd3-d43a-466d-a471-5f0aac331942 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:39.907568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:39.910499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:39.911850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 258/521] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:39.912815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:39.940115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:39.940788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:39.941087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:39.941341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:39.949809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:39.951278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:39.964151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:39.965062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:39.965942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 264/522] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:39.966429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.025657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c7ddd-cef9-4029-aed6-f729fdb0b286 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.061403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.062001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.062280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.062555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.063217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.063518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.064928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.075837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3e12-ade9-435e-a681-1b9d702cdd99 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.081652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 259/523] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:40.081652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.084505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.118068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.118662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.118937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.119240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.156323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.186661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a1ba-cd5a-4283-ad9f-abcaaf12a34e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.193043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.194535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.227331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.228083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.228650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.228960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.229956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.230264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.231890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.252017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 260/524] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:40.252648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.304799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 265/525] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:40.306206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.324928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.363616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424717f2-d846-49e8-bf51-ae9656b02c41 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.379281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.400399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.401044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.401280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.401554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.402236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.402541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.404015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.416331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fea57-1df6-4875-acd4-860fa208f74c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.427164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 261/526] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:40.427164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.459191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.459748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.460027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.460311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.467007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.468614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:40.479951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:40.480815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:40.481713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 266/527] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:40.482205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.552007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542f8e-9cc2-4451-b50e-2a9f93b71235 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.577995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.590430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.591078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.591380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.591650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.593820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ecc7c-aaaf-4729-9343-c597d6914f29 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.621167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.630804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.631489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.631943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.632235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.644447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.646469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:40.661599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:40.662738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:40.663811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:40.663811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:40.663811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:40.667209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 267/528] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:40.668020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.670723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.672426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.768624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6bf7f-1422-4be8-b626-d8f1bd0fd07a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.774630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 262/529] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:40.775826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.800150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.808238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.827334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.827798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 268/530] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:40.827798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.869640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:40.922051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff817522-464a-4859-99df-d8c160de0f00 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.940337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38715e-fb87-45af-a79b-08526346145c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:40.965950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.967452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.967840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.968174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.968965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.969320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:40.970857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:40.991416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 263/531] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:40.992992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:40.994881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:40.995901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:40.996332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:40.996714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:40.997870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:40.998289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.000810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.017746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 269/532] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:41.019176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.106574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.130840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902b2de-e372-45bf-94a9-8ea6188b300f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.136989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0456-8745-483e-ae61-73bb7f49a724 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.138583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.168724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.169532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.169911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.170294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.175811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.176791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.177207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.177637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.178501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=c1c10f44607a4d8080891c89b9b88345) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:41.178946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.180538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.182938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.184529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.187053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 270/533] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/role_assignments?user.id=c1c10f44607a4d8080891c89b9b88345&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:41.188188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:41.213413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:41.214898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:41.215960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:41.215960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:41.215960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:41.223922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 264/534] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:41.225013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.311174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c1609-edd4-4ae4-8329-0f3793a8a186 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.352473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.353885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.354306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.354724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.362703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.365536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:41.379060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:41.380721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:41.381838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:41.384118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 271/535] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:41.389617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.422242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.475949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ce9f-2759-4124-89ca-7dea499e5858 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.516033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdbb73-6b0d-46f5-9ad1-5376a20566e5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.519796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.520910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.521390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.521818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.522607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:41.523049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.524616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db2004-c510-483a-b1ef-dfa221a49040 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.527136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.540937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 265/536] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:41.546809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf392-1805-4165-b266-b7348052f5fd None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.547245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf392-1805-4165-b266-b7348052f5fd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.547586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaf392-1805-4165-b266-b7348052f5fd None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.553060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 266/537] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:41.553950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaf392-1805-4165-b266-b7348052f5fd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.559401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.560183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.560610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.561005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.561871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:41.562318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.563967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bae0a-ec7c-47b1-8bd4-674d1358d585 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.589097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 272/538] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:41.590828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.591555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.591969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.615660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:41.617791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:41.669742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaf392-1805-4165-b266-b7348052f5fd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.704849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 273/539] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:41.705850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.709118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.711043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.711537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.711961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.720390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.722566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.725165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:41.738355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:41.740060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:41.741108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:41.743985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 267/540] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:41.751239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:41.847438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67745f0f-405a-4d9e-94b1-640e9e7dbdba None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.867499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:41.890875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:41.891693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:41.892287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:41.893389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:41.894235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=33f0625dda614821a5d9ca6c75de33d4) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:41.895475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:41.900996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:41.909251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 274/541] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/role_assignments?user.id=33f0625dda614821a5d9ca6c75de33d4&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:41.951248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5c970-c6d6-498f-891d-67f19fe6f1ab None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:41.992384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.019854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.021763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.022040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.022308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.023021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:42.023349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:42.024824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:42.055260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 268/542] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:42.056808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.116391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:42.139177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991028-4d6d-4f42-aa19-f0458502b2bf None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:42.195567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.196196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.196474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.196746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.215933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:42.222135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:42.223044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:42.224995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 275/543] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:42.231686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.247652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47745e6-8c7c-427b-a4e7-5db6d210c9e9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:42.280255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:42.326269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.327022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.327465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.327752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.328481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d40b4a33e1e845c4b5faaf0b1e608c59) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:42.328796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:42.331405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:42.345036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 269/544] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:42 2022] GET /identity/v3/role_assignments?user.id=d40b4a33e1e845c4b5faaf0b1e608c59&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:42.370177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36441654-d2f8-48bf-97f6-fea31add2185 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:42.414731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.415373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.415657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.415929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.416605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:42.416908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:42.418277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:42.428135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 276/545] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:42.434021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.607372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415e8c5-bc54-48e8-9b04-b6962a6956b1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:42.655502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.656055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.656339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.656616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.671453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:42.672926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:42.687828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:42.688762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:42.689975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:42.689975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:42.689975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:42.689975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 270/546] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:42.700196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.703225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:42.831319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29643827-23dd-4356-8bea-831b3698d62a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:42.891066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:42.891695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:42.891969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:42.892237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:42.892872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:42.893166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:42.894525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e7f5b-8473-4991-99ba-0b71907f20e8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:42.912827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 277/547] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:42.918523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:42.968432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.043226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34212e9-d116-4dc4-b197-bf08c1a2d619 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:43.046688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887724d-bc51-45aa-93ec-b06cd7833912 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.046818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887724d-bc51-45aa-93ec-b06cd7833912 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.047051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887724d-bc51-45aa-93ec-b06cd7833912 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.048636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 278/548] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:43.054000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.054591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.054591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.073396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:43.074569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:43.125924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:43.126642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.127008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.127359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.144279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:43.144592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.147345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:43.157015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 279/549] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:43.168024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:43.168961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:43.169887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 271/550] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:43.174807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:43.315917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a8818-d2b3-44f5-97b5-de9fdca25be4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:43.362114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:43.363082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.363580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.364291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.365256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:43.365757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:43.367827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:43.392322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 280/551] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:52:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:43.396701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:43.490911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:43.543353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.553945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.556859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4105df3-2d1b-489b-a7b3-ce787a52f3ae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:43.601123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:43.602050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.602488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.602916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.677101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad5a6e-ca25-491b-a1ef-a85521ad288c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:43.677623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:43.679397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1f88b-d9ca-4ed2-8caf-67b65966f7a3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:43.749369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:43.749921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.750233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.750722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.766816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-a73a2573-ab46-4a27-9596-5845cb58bf00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:43.775012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:43.776204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:43.777033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 281/552] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:43.777810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0d23c-9c2d-4947-8638-151f8af33622 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.778010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0d23c-9c2d-4947-8638-151f8af33622 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.778236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0d23c-9c2d-4947-8638-151f8af33622 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.780305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 282/553] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:43.783984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0d23c-9c2d-4947-8638-151f8af33622 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:43.786617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.798241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 272/554] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:43.799845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.800246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.800633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.824717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:43.826336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:43.908273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 273/555] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:43.909233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:43.915730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0d23c-9c2d-4947-8638-151f8af33622 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:43.939580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:43.989972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:43.990695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:43.990995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:43.992154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:43.992828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:43.993132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:43.994498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.017473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 283/556] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:44.020332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.106524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db0c85-4af5-4196-8052-9e6e04d40a3e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.148006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.166344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.166982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.167303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.168517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.192915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.194402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.226167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f7763-bcdf-4e3b-98d1-baf7ee343a99 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:44.226806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.227821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:44.228696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:44.228696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.228696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:44.230122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 274/557] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.235228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.271930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.271930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.271930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.271930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.283259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.285105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:44.302345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.303926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:44.304840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:44.304840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.304840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:44.304840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 284/558] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.304840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.337284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.356674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431fac2-6dcf-4f18-9888-7a892059498a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.391866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.392488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.392762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.393028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.403356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-516182d8-9c5b-4823-b65e-1110acbaeac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:44.408613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.409537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:44.410481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 275/559] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.412446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120bb8-a75c-4796-ad61-c355e2ea2bf1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.412446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120bb8-a75c-4796-ad61-c355e2ea2bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.412446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120bb8-a75c-4796-ad61-c355e2ea2bf1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.417297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 276/560] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:44.417297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120bb8-a75c-4796-ad61-c355e2ea2bf1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.423299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809cc6-62ef-41d6-a3f4-6688718391ef None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.451149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.459629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.465224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.476394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 285/561] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:44.477215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.530920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120bb8-a75c-4796-ad61-c355e2ea2bf1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.571195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.573281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.573997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.574369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.574713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.575461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:44.575832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.577267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c456-1a87-4d39-b981-b6ecd87ea076 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.587671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db67ee3-6ba4-41f0-82bf-fa22857d4bdc None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.594826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 277/562] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:44.594826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.594826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.594826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.611535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:44.612747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:44.626492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.627090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.627397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.627678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.628314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:44.628612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.630541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.638973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 286/563] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:44.639727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.686991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 278/564] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:44.687849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.711843 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.749924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7663cc-4d33-4422-9449-890c88dc1a81 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.784372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.784941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.785197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.785723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.792953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.794369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.798275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7eae2-cbe7-4331-a3f6-410f3cf5564f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-8ea0ba86-406d-418f-95b5-47c816985a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:44.806781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.807869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:44.808791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:44.811417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 287/565] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.813226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97eb5-500e-4806-a4f8-e59333d4a415 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.813797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97eb5-500e-4806-a4f8-e59333d4a415 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.814077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b97eb5-500e-4806-a4f8-e59333d4a415 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.815939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 288/566] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:44.816507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b97eb5-500e-4806-a4f8-e59333d4a415 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.826338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.835133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.835759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.836149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.836317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.843518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.844932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:44.857159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.858125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 279/567] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.863653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:44.929826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b97eb5-500e-4806-a4f8-e59333d4a415 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.948185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:44.965702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca21f4-da5b-47a3-ba72-408916733c15 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:44.967276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:44.967871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.968151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.968420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:44.979821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:44.981256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-04f26bc1-9f09-43b2-90eb-bbc829567665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:44.996380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:44.997301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:44.998205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:44.998205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:44.998205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:44.998205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 289/568] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:44.998626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:44.998921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:44.999150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.004512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.008527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.008527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.008527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.008527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.008527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.008954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.017882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:45.019047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:45.021559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 280/569] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.022192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.077442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.090958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 290/570] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:45.091790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.157175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29af96-23ff-4cf8-820f-3bca5b95fdd1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.196174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c87aa-20f4-47f9-8c3d-928e0a9a7208 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.196933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.198011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.198304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.198595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.199969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.200457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.202116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.217772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.225743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 281/571] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.226594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.233112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.233746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.234022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.234292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.245843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:45.250792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:45.251789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 291/572] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:45.252674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.305597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.340772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763e82-3164-46a4-8ad9-e9825e85ab2a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.359537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5628410-5f00-4e8c-aab6-6dc4fbd0a1ad None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.376085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.376686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.376987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.377239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.377903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.378206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.379614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.391540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 282/573] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.392325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.394440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.395042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.395374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.395651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.396367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=0c95310969054beab60e99fd71a34aee) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.396676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.398082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.403611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 292/574] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/role_assignments?user.id=0c95310969054beab60e99fd71a34aee&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.404242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.474588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.517002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a4e02-bbaf-43d6-8689-8dfabfadbb75 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.523968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a975e4-3b58-4a04-9a2e-0f78aee01c65 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.531720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.560223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.560888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.561175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.561449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.562341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.562653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.564096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.571805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.572647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.573069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.573484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.573588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 293/575] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.574300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.589264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.591127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:45.607690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:45.608838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:45.609922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:45.609922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:45.609922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:45.611302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 283/576] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:45.612216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.692749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31e002-9600-471c-9a45-e457b32016dc None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.722418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e824e61-6164-4117-9d22-35f3d837dbcd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.728371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.729112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.729492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.729854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.737343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.742122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:45.747463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:45.748583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:45.749636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:45.755533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 294/577] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:45.755533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.765724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.766424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.766703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.766971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.775606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.778117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.789006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:45.790252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:45.791348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:45.792443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 284/578] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:45.793093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.868698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7fca1-9106-4384-b12b-7b9145a2099b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.899934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13960098-3c0c-4a28-b3b0-a6ac82bf785c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:45.906227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.906857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.907586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.907864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.919493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.920934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:45.937274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:45.938295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:45.939238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:45.939238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:45.939238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:45.939238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 295/579] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:45.939238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.944499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:45.945108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:45.945389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:45.945662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:45.946302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:45.946603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:45.948023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:45.965587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 285/580] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:45.965587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:45.995389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.033943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.051311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70aa9f9-6817-4c00-b05c-9bc07513401b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.097429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.098033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.098307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.099298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.099951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.100248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.101422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a8a84-9f74-4dfe-b561-c7b640c205af None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.101609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.111448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 296/581] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.112991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.140686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.141220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.141496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.141767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.142533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.142840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.144313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.167618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 286/582] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.168474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.222196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4203d28-78d9-4e7f-9ca5-6f91de47a58b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.258550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.259177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.259455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.260146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.260903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.261182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.262530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.262732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.272768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ecef8-e7e0-4751-b65d-39d15af1b42d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.275530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 297/583] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.282413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.293011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.311045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.311694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.311973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.312248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.319537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.320990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:46.332349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:46.333930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:46.335054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:46.338788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 287/584] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:46.339581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.397097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7318-b367-4cc6-8de6-e61ad83582f5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.433462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.434082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.434355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.434621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.441338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c49b-27ce-4819-94f1-8b39b97c8d3a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.445869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.447330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:46.463616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:46.464645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:46.470604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:46.470604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:46.470604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:46.470604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 298/585] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:46.470604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.486946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.487712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.488002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.488276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.498789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.500270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:46.515576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:46.516575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:46.517466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:46.517466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:46.517466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:46.517466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 288/586] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:46.517466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.522040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.538854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.589885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ef8fe-51fe-4cb5-8e89-478721d47552 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.624928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3c979-bea5-4bfb-9fa8-c0ea3d0486c9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.633040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.633655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.634283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.634554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.642882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.644360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:46.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:46.656916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 299/587] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:46.657874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.662649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.663276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.663553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.663823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.664502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.665525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.666222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.683237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 289/588] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.683237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.749128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.773278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ac7fe-4f2c-4a90-94c6-bc25b231fe15 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.794894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f519e9c-a8b5-4bda-86a7-8bf304d6cebf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.795064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:46.809245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.810126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.810126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.810444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.811087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.811416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.812780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.825649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 300/589] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.827361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.838611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.839352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.839636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.839907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.840551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.840861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.842215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.854925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 290/590] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:46.855799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:46.941899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876c917-d433-497a-8dd5-edcd5bb186ca None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.966075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0d3f-9a02-4557-9346-6fdb1d8ee6a0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:46.978137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:46.978741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:46.979025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:46.979365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:46.980170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:46.980598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:46.982026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:46.990428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.001929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.007177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.007177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.007422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 301/591] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:47.007517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/304d887d3553492bbb31f573a7952646` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.007613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.022665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=d132894fe36047418e5cd8bebe04c387, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.024139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.046807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.091153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 291/592] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:46 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:47.091153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.125200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e363088-3111-4c58-85d9-55d204c5f383 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.178296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.179798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.179985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.180254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.191216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.193415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.209100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c6147-b47e-4d70-b1d1-28d30bfb4fb0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:47.211356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:47.212487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:47.213472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 302/593] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:47.214075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.271330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.272138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.272521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.272895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.282560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.284207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:47.298729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:47.299895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 292/594] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:47.300964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.338733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.346677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.351888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdee08d-07d1-425b-ba85-9bbae37aa8bd None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.387553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.388186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.388464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.388738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.400071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.401670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.409310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4921-b663-4a96-895a-31a41ffe1b79 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:47.418263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:47.419694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:47.420683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:47.420683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:47.420683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:47.423420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 303/595] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:47.424720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.451071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.451631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.451909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.452182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.452915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:47.453214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.454653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b5629-013f-422f-8aca-0d11645fa0ee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.475245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 293/596] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:47.476786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cbdfc-2e8a-4a27-be87-f8b6afde05a7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.477071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cbdfc-2e8a-4a27-be87-f8b6afde05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.477344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cbdfc-2e8a-4a27-be87-f8b6afde05a7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.482857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 294/597] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:47.482857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cbdfc-2e8a-4a27-be87-f8b6afde05a7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.542219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187512f0-9e05-4c83-a8b8-fc7aa1d31848 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.581139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.581751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.582026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.582297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.583014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:47.583342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.584770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.597585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 304/598] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:47.598424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.608300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.613077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.617547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cbdfc-2e8a-4a27-be87-f8b6afde05a7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.653712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.659128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090ba4-0c5f-4ace-aa85-67ffe4a7a7d3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.669526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 295/599] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:47.670793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.671080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.671390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.691831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:47.693607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:47.716457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e924207-e946-41fa-8fc8-6c5143edd08b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.753814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.755852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.756453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.757036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.762770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 296/600] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:47.763623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.776523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.778664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:47.798793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:47.799876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:47.801055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:47.801055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:47.801055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:47.805832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 305/601] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:47.806216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:47.868689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.871966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:47.891223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdedc45-9563-418e-9385-1b74c3bbd9d4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.914118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24978-3d26-4075-bb40-96e34154e53c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:47.927563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.928415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.928869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.929302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/c1c10f44607a4d8080891c89b9b88345/roles/304d887d3553492bbb31f573a7952646` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.944245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=c1c10f44607a4d8080891c89b9b88345, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.946115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.949435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:47.950136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:47.950485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:47.950964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:47.957901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:47.959545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:47.972001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:47.972907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:47.975124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:47.978655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 306/602] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:47.979505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.023875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 297/603] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:47 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/c1c10f44607a4d8080891c89b9b88345/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:48.025765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.110209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb03ba2-ffa5-4910-bd2b-0ee9b12b3e4d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.138259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.151409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.153217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.153217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.153217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.153469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:48.153697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.155124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.169713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 307/604] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:48.170804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.185878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c382b-8339-4885-a8f5-39cf041c7321 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.225156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.235727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.236685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.236968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.237244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.238196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:48.238931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.241598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.259376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 298/605] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:48.260279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.302726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdaab3-47d9-4b85-b821-ccc559a6b65a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.346480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.347626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.347902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.348169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.361474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:48.368685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:48.369849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:48.371299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:48.375161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 308/606] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:48.375969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.394341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c671-bcb0-4ad3-a694-7d7f4f6de070 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.433145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.443071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.443822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.444098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.444366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.451560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.456468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:48.464752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:48.465765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:48.466686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 299/607] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:48.467204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.495527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa466-d91b-49cb-bf1d-5ff61095917c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.522744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.537356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.543155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.543155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.543155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/33f0625dda614821a5d9ca6c75de33d4/roles/304d887d3553492bbb31f573a7952646` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.557793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=33f0625dda614821a5d9ca6c75de33d4, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.559920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.579054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17dc91-92ce-4b10-a9e2-f406d5b89152 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.614425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.615299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.615299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.615714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.616475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:48.617040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.618438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.632549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 300/608] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:48.632549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.670718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 309/609] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:48 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/33f0625dda614821a5d9ca6c75de33d4/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:48.672072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.818179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d138-f2b8-4cab-85f2-7bb83898c52c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.843485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.856685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:48.865489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0372ec-6e1d-43a1-8e6c-edc27a4c3125 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:48.869002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.869002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.869002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.869002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.869364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:48.869364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.869697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.886628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 301/610] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:48.893539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:48.917182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:48.918482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:48.918778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:48.919050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:48.933569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:48.935035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:48.955976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:48.957015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:48.958746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:48.958746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:48.958746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:48.958746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 310/611] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:48.963882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:49.030783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f3bd-0d11-478a-b788-1506ff8ec246 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:49.093361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:49.094000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.094280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.094597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/d40b4a33e1e845c4b5faaf0b1e608c59/roles/304d887d3553492bbb31f573a7952646` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.124733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=d40b4a33e1e845c4b5faaf0b1e608c59, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:49.127568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dbca-46eb-44ec-9d5c-cb93f5f97029 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:49.129340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf4390-540d-47f7-9ca0-830bb50b69b7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:49.170306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:49.170951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.171276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.171550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.172194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:49.172548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:49.174044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:49.187502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 311/612] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:49.193109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:49.197189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:49.228122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 302/613] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:48 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/d40b4a33e1e845c4b5faaf0b1e608c59/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:52:49.239239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470610e-d896-45e7-ac42-687fceb96084 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.239756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470610e-d896-45e7-ac42-687fceb96084 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.240432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470610e-d896-45e7-ac42-687fceb96084 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.244905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 303/614] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:52:49.250849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.260254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.260254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.299969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:49.302278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:49.335480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:49.392189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4efa7-7b42-4e40-b53c-92fc5b85ff3d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:49.437663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 304/615] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:49.453237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:49.453871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.454146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.454418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.463221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:49.464779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:49.477871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:49.478871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:49.479889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 312/616] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:49.488217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:49.607259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada970df-18cd-43fd-aa7c-aa4fe9105ff6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:49.643774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:49.645032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.645502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.645935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.646812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:49.647281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:49.648888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:49.668083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 305/617] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:52:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:49.676196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:49.769848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:49.816688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a123d3-ed60-4c4d-b3c0-39c6ee84bb22 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:49.854764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:49.865353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:49.865988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:49.866267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:49.866536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:49.877754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:49.890644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:49.893852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:49.917391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:49.918382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:49.919455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:49.919455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:49.919455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:49.919455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 313/618] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:49.924255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.070056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb9046-687e-4bde-a0c4-6fc82423e2b0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.080949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c960ec-cc55-4cf5-9429-5c59a5e48ff0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.098284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.121664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.122539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.122816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.123085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.135460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.136064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.136344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.136475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:50.136610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.137319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:50.137624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.138980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-32a870e1-4a07-4363-8ed6-716c8417b89d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:50.142901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:50.144088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:50.145159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:50.148109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 306/619] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:50.150308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b120-e79f-4214-8978-a79c6e065b77 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.150739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b120-e79f-4214-8978-a79c6e065b77 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.151196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29b120-e79f-4214-8978-a79c6e065b77 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.152800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 307/621] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:50.153741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b120-e79f-4214-8978-a79c6e065b77 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.159464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 314/621] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:50.160395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.235688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.271963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29b120-e79f-4214-8978-a79c6e065b77 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.287149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac24cc-2063-4bf9-bac1-71bd9bb8e656 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.309386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.310275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.310752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.311260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.312097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:50.312595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.314149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5afd-439a-40b2-8c2a-78127c0473ee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.324059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 308/622] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:50.325758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.326263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.326752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.328905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.329529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.329803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.330070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.338669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.340415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.349148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:50.350428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:50.353952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:50.354864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:50.356104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:50.362827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 315/623] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:50.363947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.410591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.435199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 309/624] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:50.436379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.491661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec58f3-a640-4a33-a82c-6ded0d5a81e0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.535138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.535138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.535138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.535138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.545567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.547507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:50.564080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:50.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:50.566024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:50.566024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:50.566024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:50.566024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 316/625] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:50.571009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.589566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.659043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e96a3-f148-47e1-9c66-98d8eb2857c5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.695590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.708813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c481-d6b2-4cc4-8845-a46c757dff98 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.731282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.731282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.731282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.731282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.731638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:50.731638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.731638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.746169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.746837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.747128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.747404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.748198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:50.748498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.749852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.755664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 310/626] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:50.761603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.771672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 317/627] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:50.781829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.876575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78ada-9c2b-4824-bca7-059aa377deae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.892127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f792d1-1961-45b8-93b3-2afe7b728566 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:50.901658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:50.913906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.914557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.914835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.916873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/0c95310969054beab60e99fd71a34aee/roles/304d887d3553492bbb31f573a7952646` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.928417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:50.929040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:50.929324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:50.929592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:50.929988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=0c95310969054beab60e99fd71a34aee, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.931464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6428ccf-cd6a-4785-982d-1b4fe03ee249 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.937566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:50.938976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:50.951011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:50.952111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:50.953090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:50.955249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 318/628] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:52:50.956327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:50.966933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.003184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 311/629] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:52:50 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/0c95310969054beab60e99fd71a34aee/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:52:51.004763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f19a-bf75-43e6-b36d-c8557d9c8ede None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.005081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f19a-bf75-43e6-b36d-c8557d9c8ede None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.005355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f19a-bf75-43e6-b36d-c8557d9c8ede None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.006793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 312/630] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:51.011458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6f19a-bf75-43e6-b36d-c8557d9c8ede None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.075616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d955-14a2-4b2b-844a-4bd9e7a565d9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.115735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.116376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.116647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.117167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.136271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6f19a-bf75-43e6-b36d-c8557d9c8ede None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.137100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2f16749b-a1a6-4059-a6d7-6805af52af48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:51.145278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:51.146353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:51.147249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:51.154001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 319/631] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:51.154001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.154001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.154001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.176339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:51.178197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:51.193639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.194639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.195310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.196209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.197229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:51.197761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.199889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.221591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 313/632] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:51.222735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.258744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.259996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.266978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 320/633] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:51.271905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.338104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dcaa2-bbaa-457e-8dbd-e12533211904 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.375736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.376877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.377430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.378001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.378884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:51.379420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.381679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.395345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 314/634] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:51.397284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.443061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b962c-efd7-48df-b033-c0df793f6b41 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.505762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13671ad-2dc5-4f38-8547-616a288099fa None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.515830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.516453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.516734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.517005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.519611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.545377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.546520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.547005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.547612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.561869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.563782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:51.580192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:51.581221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:51.582168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:51.582168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:51.582168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:51.588177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 315/635] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:51.590457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.666017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.674402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.675930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb374-a90a-4481-b563-7dba1cb55871 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.712203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797df1c2-a93c-4b07-947b-53a132298a49 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.749794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.750389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.750669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.750943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.751613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:51.751918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.753287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.770175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 316/636] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:51.770175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.773144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.812458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 321/637] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:51.814363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe962b-7a0b-44d6-a6a9-5d2ba452545b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.814787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe962b-7a0b-44d6-a6a9-5d2ba452545b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.814984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe962b-7a0b-44d6-a6a9-5d2ba452545b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.816515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 322/638] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:51.817879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe962b-7a0b-44d6-a6a9-5d2ba452545b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.881124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725681f-9ef2-4b01-9ebb-6379f8e826fb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.917690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.918551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.918837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.919184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.919332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:51.923750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe962b-7a0b-44d6-a6a9-5d2ba452545b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:51.930312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:51.935377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:51.936533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:51.937780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 317/639] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:51.940270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:51.966612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:51.967542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:51.967823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:51.968095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:51.978891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:51.980377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-48f85570-d0b9-4876-8890-536b9305d6e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:51.996644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.002430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 323/640] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.003762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.004644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.008764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.027400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:52.029244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:52.046846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747fbf2-257f-4d89-bfdb-34705de5bf26 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.083576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.084313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.084846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.085384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.095855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.097388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.115128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 324/641] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:52.115128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:52.117897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:52.120194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:52.120194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.120194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:52.121887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 318/642] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.122728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.223250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.232300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ced75-958d-408b-89a4-f2fb57991cd8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.242266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a62bb3-31c6-42a0-a705-c54900beb14d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.255825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.269885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.270867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.271187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.271466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.272314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:52.276111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.276111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.280506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.281293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.281762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.282292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.283137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:52.283588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.285090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.293754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 319/643] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:52.294517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 325/644] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:52.299072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.299412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.404585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e6755-a3eb-4879-9554-fa46371f9ffd None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.405421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231b91-d4de-43bb-af62-f2bc2359ff8d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.440795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.441403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.441686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.441961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.443432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.444024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.444304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.444575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.445208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:52.445511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.446904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.459438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 320/645] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:52.460082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.467316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.496257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.519927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.522133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.569161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dce32-53dd-44b6-9d72-21eef92f0959 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.604855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.605480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.605761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.606035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.618806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.620344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:52.635695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.636731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:52.642739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:52.642739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.642739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:52.642739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 321/646] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.642739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.643045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 326/647] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:52.643605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.696774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.732806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.748495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf61e89-a332-4b0b-8ca6-22f9af374369 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.750874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb06e5-641a-41c0-b7b3-d8b09b778105 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.785787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.786961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.787319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.787461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.787654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.787938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.788214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.788484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.794430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.795905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.798980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:52.804859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.805826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 322/648] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.811806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:52.812595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.814511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 327/649] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.815440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.913892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc800-28f5-454a-b914-a7a075ccf1b8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.924219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:52.925858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4911deb-a6fb-433e-ac58-b18b7aca1a6e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:52.948961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.949649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.949925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.950199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.961058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.962575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.963845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:52.964606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:52.965040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:52.965313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:52.965953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:52.966252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:52.967703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:52.977219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:52.978217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:52.980169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:52.980169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:52.980169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:52.980169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 328/650] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:52.980169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.984577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 323/651] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:52.985656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:52.997740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.085590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf661181-9c46-4524-bbdd-19959f6187bb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.095305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e6b2-67b3-4381-a7c2-d29f4cf9699c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.120220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.120826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.121106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.121378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.122034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.122335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.123740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.129868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.130547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.130823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.131093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.131560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 329/652] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.131801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.132102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.132343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.133522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.144821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 324/653] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.145729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.163871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.208184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.237262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80c8d4-8c1a-496c-be87-377c2389848a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.243162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5483270-fab0-4912-9385-48b719263ec3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.270994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.271622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.271907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.272174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.272809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.273746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.275138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589fc2-86ed-44cf-b483-ed545b8a3a74 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.277111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.277736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.278017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.278288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.285714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.291051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 330/654] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.291211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.291301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119000cd-ca97-4496-afbd-04f73810f147 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.291301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119000cd-ca97-4496-afbd-04f73810f147 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.291301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119000cd-ca97-4496-afbd-04f73810f147 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.291301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 331/655] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:53.291301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119000cd-ca97-4496-afbd-04f73810f147 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:53.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:53.303476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 325/656] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:53.304519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.381873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.398593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119000cd-ca97-4496-afbd-04f73810f147 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.402563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8588c-de8b-4696-82d2-6fd54471ccdd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.427556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.432522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.433228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.433509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.433783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.436505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.440251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.444605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.446136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.446382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 326/657] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.447190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d2a7eb91-7ffd-40f8-9c14-540a904c1d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:53.472648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:53.473771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:53.474679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:53.474679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:53.474679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:53.474679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 332/658] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:53.475807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.476090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.476364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.498439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:53.499714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:53.575919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb21a9c-5ea5-4e67-ad57-aa0c86e5ff85 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.581869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 333/659] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:53.587916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.628258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.628875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.629149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.629417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.637913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.639734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:53.652346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:53.653264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:53.654179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:53.656066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 327/660] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:53.656981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.682122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.724991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217838cc-4b2d-4e41-bc29-b8cfbf19c97c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.725268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.773322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.774484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.774765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.775039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.776078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.776634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.778793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.787091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e718f-7f8a-4246-b39b-41c9b5ee4b6c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.807196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 334/661] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.808346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.828720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.829327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.829611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.829883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.830528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.830830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.832269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.845712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 328/662] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.846549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.928563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132ee68-4523-40aa-8b8e-f9067fc06a15 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.954292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.958849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dd2a0-ec81-4742-b205-0afbde1133ec None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:53.968076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.968970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.969422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.969568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:53.970390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:53.970580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:53.971622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:53.972020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:53.989174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 335/663] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:53.990206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:53.996500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:53.999062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:53.999062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:53.999062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.071594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.073246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.099351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345b93c-2f63-41d9-ba67-f01014dfb21b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.133517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.134143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.134425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.134696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.142181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.143649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:54.155733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:54.156705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 336/664] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:54.157705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.185356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 329/665] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:54.186379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.201097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.213856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.272933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd321ea-563a-441e-9a54-28d9b19d252f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.291825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0ff9c-04ab-47d5-a85c-dc3d4f67e87f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.308506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.309417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.309926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.310337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.321552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:54.326475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.327141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.327431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.327701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.328441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=c1c10f44607a4d8080891c89b9b88345) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:54.328747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:54.329251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:54.330352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:52:54.331464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:54.332257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.338263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 330/666] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/role_assignments?user.id=c1c10f44607a4d8080891c89b9b88345&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:54.338263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.338622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 337/667] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:54.339663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.450771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.479304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b2e-6dff-416f-9069-7ac4bd218a42 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.491140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ebe9f-1f96-4830-a17e-57f3a1956e04 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.493986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.529068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.529680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.530831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.531143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.531872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:54.532176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.533759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.539696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.541335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.541824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.542225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.561336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 331/668] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:54.563175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.569255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.570748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:54.590261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:54.591498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:54.592401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:54.592401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:54.592401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:54.592401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 338/669] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:54.592401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.701351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ff30-3b06-47dc-b995-7c27e839f875 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.714479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249e72-d506-4fdd-b572-030f9881696d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.740851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.749297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.749897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.750176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.750445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.751129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:54.759973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.759973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.766925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 332/670] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:54.768169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:54.775083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.775751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.776030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.776302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.810723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:54.866538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.868006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.904586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa693-56e6-4b93-a244-dbadb68bc011 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:54.952687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:54.953542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:54.953920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:54.954290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:54.955160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:54.956340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:54.957319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:54.978412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 333/671] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:54.979836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.000985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 339/672] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:55.000985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.057430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.080809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.107790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1646ca-78f3-4b54-8e8f-98ef19503e35 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.139871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7382cb-f88f-4177-945a-c78cb192cff9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.146702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.147374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.147659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.147935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.159762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.161250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:55.179046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:55.180188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:55.181094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:55.181094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:55.181094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:55.185305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 334/673] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:55.186291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.190971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.191920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.193348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.196408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.204869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.206588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:55.227458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:55.228464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:55.230391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:55.232941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 340/674] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:55.233939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.302999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49cb8-8698-4ee2-b9aa-8d95e7598fc8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.315657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.340035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.340839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.341704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.341704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.354251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.355917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.365499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901e652-49b9-4240-be05-cc931d22669b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:55.374386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:55.375428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:55.376365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:55.376365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:55.376365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:55.381680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 335/675] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:55.383387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.389533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.409278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.410205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.410768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.411746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.412561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:55.413032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.414940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.435216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 341/676] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:55.436410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.516428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a57606-cd73-4da9-9c0f-22bbbd072af7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.542304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed5965-b1fb-440f-9450-34f5caea9dd8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.565736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.566358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.566626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.566964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.568795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:55.569110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.570581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.586213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.587203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.587813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.588285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.589227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:55.589862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.591503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 336/677] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:55.591746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.592423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.605353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 342/678] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:55.606504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.622746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.643529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.711211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97622-f1e4-4690-9975-77e5b5eb3488 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.717535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20198d-d245-4f9d-b0cc-424eca957f78 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.749990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.750952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.751266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.751540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.755963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.756563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.756842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.757141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.757866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=33f0625dda614821a5d9ca6c75de33d4) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:55.758176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.759419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:55.759686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f46ac4-4cc6-41bc-8d42-c0055787c73e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.760870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:55.771286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 343/679] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/role_assignments?user.id=33f0625dda614821a5d9ca6c75de33d4&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:55.771286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea98a-0fab-411d-8df5-4cb033a2c71e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.771286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea98a-0fab-411d-8df5-4cb033a2c71e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.771286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daea98a-0fab-411d-8df5-4cb033a2c71e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.771286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 344/680] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:55.774283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daea98a-0fab-411d-8df5-4cb033a2c71e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3ce6cd62-19d5-4951-945b-a1712a3d28e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:55.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:55.781129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:52:55.782098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:55.789220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 337/681] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:55.790874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.791372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.791811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.812064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:55.813550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:55.898411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 338/682] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:55.899669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:55.909009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.920177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daea98a-0fab-411d-8df5-4cb033a2c71e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:55.920390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:55.980692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:55.981852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:55.982374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:55.983041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:55.998912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.001513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:56.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:56.019260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:56.020408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:56.023476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 345/683] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:56.031136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.054459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f5d38-8a26-4a12-957b-33fca124a156 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.100619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.101249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.101777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.102053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.103375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:56.103792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.105850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.129119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 339/684] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:52:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:56.136258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.158786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dacaab-fb52-441b-9cc6-3ee68700b74d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.213854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.214496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.214773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.215181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.215975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:56.216282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.217740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.222291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.233222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 346/685] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:56.234127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.260179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1133-b822-49c9-88e8-51de8db16d3e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.260852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.300431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.301028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.301315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.301686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.361493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd428b8-d884-40ba-9e34-6cc17f1c9721 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.379806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.381845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.405989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.406642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.406920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.407220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.420663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:52:56.426696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:56.428193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:52:56.429067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:56.431180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 347/686] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:56.432271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.459955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.501866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 340/687] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:56.503372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.538883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.569989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbaf7b-5395-4fe0-a390-e652b7d39335 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.616745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.618108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.618558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.618973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.620004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d40b4a33e1e845c4b5faaf0b1e608c59) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:56.620775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.622775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.629873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0682cc-66df-4a97-81dc-2ae66230fcff None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.630782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 348/688] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/role_assignments?user.id=d40b4a33e1e845c4b5faaf0b1e608c59&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:56.631933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.679791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.680736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.681192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.681605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.682439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:56.682893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.685506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac7b3b-d8a0-476d-ba91-da278b9373e7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.703034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 341/689] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:56.706038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b96c4-e58c-46fb-936e-0489cdb05bdc None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.706559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b96c4-e58c-46fb-936e-0489cdb05bdc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.707232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b96c4-e58c-46fb-936e-0489cdb05bdc None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.709936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 342/690] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:56.711290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b96c4-e58c-46fb-936e-0489cdb05bdc None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.755976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.772204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f07d-81c5-41b5-a397-f8965dc02566 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.817924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.818789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.819261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.819692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.832548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.834344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.837488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-ffd9ae63-207d-4c70-a40f-24e8411aa506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:56.854253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:56.855356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:56.856480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:56.856480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:56.856480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:56.856480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 349/691] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:56.856837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.857110 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.857455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.861828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b96c4-e58c-46fb-936e-0489cdb05bdc None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:56.879811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:56.880991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:56.914841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:56.915821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:56.916102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:56.916374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:56.928718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:56.929977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:56.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:56.946338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:56.947293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:56.947293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:56.947293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:56.947293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 343/692] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:56.951895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:56.965429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 350/693] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:56.966746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.051156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.070727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee2d91-fc94-4281-9cc5-34f9a94aad12 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.133263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4ae09-6c4a-41c6-9de2-8e06f73fa603 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.140323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.141153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.141739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.142159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.142953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.143425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.144960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.175267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 344/694] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.180058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.182334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.188053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.188681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.188958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.189229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.189873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.190236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.191721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.204901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 351/695] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.214298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.323859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a1c4-56a4-46bb-b0f7-4a9fe0d41f6a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.361905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.362724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a930d5c-7a49-4323-be05-df83e5dcd408 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.362850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.363207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.363675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.372949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.374584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:52:57.387584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:57.388573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:57.389563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 352/696] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:57.390169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.406128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.407084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.407613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.408089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.415369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.417162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.420071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:57.431619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:57.432557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:57.433496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:57.435884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 345/697] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:57.437037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.449399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.513419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b5753-694b-479b-99e2-35503ff8a8b6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.543732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6f32d-991a-4769-85d3-481337afa229 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.550831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.552121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.552392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.552665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.565100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.566675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:57.583559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:57.584745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:57.585651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:57.585651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:57.585651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:57.589454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.589608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 353/698] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:57.590420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.590654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.591415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.592016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.592962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.593471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.595208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.615231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 346/699] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.616401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.679522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.700237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.704575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1e05d-4580-4209-9b06-ead66499fdb2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.721197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2817-956c-41ea-9e8f-66eb4b15755a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.745187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.746043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.746431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.746802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.748095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.748535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.750444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.763523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.764683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.765148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.765624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.766447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.766930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.768586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.771212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 354/700] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.772891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.790928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 347/701] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:57.905968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb885a0-a524-4a1b-8bc7-1db1eac73dc9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.924759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9af63-164c-490c-b378-5969600a54f2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:57.947258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.950091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.950801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.951174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.951526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.969341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:57.970502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:57.970967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:57.971465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:57.972532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=0c95310969054beab60e99fd71a34aee) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:57.973424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:57.975410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:57.987882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:57.996433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 348/702] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/role_assignments?user.id=0c95310969054beab60e99fd71a34aee&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:57.996433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.068113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.069822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670faa16-19ec-48a6-97ff-2decb0f8d611 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.167365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58070b4c-b130-4c54-b285-2fb131983570 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:58.193629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 355/703] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:52:58.194991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e115ad-c5a3-48b0-8cd6-39d7d6136e38 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.195303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e115ad-c5a3-48b0-8cd6-39d7d6136e38 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.195579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e115ad-c5a3-48b0-8cd6-39d7d6136e38 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.197101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 356/704] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:58.201784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e115ad-c5a3-48b0-8cd6-39d7d6136e38 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.210013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:58.210624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.210909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.211231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.211872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.213323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a5e92-6fbc-4933-a587-221c3f1e2ac3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.273707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:58.298573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 349/705] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:52:57 2022] POST /identity/v3/users => generated 254 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:52:58.300804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.301153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.301457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.327958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:58.329198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:58.330225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e115ad-c5a3-48b0-8cd6-39d7d6136e38 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:58.367828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:58.368470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.368743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.369009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.371669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:58.382330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.383991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:52:58.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:58.403709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:52:58.404643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:52:58.404643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:52:58.404643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:58.404643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 357/706] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:58.412520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.437072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 350/707] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:58.609458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7385c-2adb-4670-b2e8-0b6913b60556 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:58.621116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:58.647066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:58.647698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.647973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.648240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.648878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:58.649174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.650603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.663179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 358/708] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:58.668872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.778354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.785060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aa01c-c920-4da0-9651-bd9c399e1620 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:58.830919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:58.831793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.832036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.832432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.840030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.841583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:52:58.854081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:58.855048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:52:58.855987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:52:58.859227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 351/709] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:58.861206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.896406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49ba68-bc75-4ba0-a6b4-1b84ca7ca185 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:58.898952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:58.934461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:58.935089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.935392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.935660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.936230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:58.937591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-9087ef63-b7f9-405a-9b17-49e0ade460f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:58.953118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:58.954277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:52:58.955217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:58.957057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 359/710] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:58.957875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b92f-361e-48de-831c-b2d934af63ed None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:58.959745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b92f-361e-48de-831c-b2d934af63ed None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:58.960535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6b92f-361e-48de-831c-b2d934af63ed None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:58.962512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 360/711] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:52:58.964223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6b92f-361e-48de-831c-b2d934af63ed None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:58.990032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6459312-da32-4279-bcfe-aef79f9d03a3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.005004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:59.034911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.035476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.035750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.036047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.037229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:59.037535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.039293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6284cc-1c28-4bce-a09e-3919e21e968a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.059139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 352/712] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:52:59.059139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.059139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.059139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.080678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:52:59.082204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:52:59.087325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6b92f-361e-48de-831c-b2d934af63ed None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.135614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.136223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.136497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.136771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.137654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:59.137960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.139377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.146087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 361/713] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:52:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:59.147001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:59.163632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 353/714] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:52:59.164906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:59.185566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:59.275253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1340644-7b11-4ba5-94dd-d32fe921c38c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.279530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:59.283492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fdaec-ad5b-4441-a9b0-02c5e2a4582e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.321444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.322410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.322897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.323464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.324411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:59.324934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.326606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.331251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.332138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.332422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.332694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.333866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:59.334164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.335604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.335841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 362/715] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:52:59 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:59.352043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 354/716] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:52:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:59.574522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:59.678295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88275-7724-45db-b7da-eccde74ad853 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.714470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.715480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.715957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.716425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.717203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.718789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:52:59.729076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:52:59.730097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:52:59.730932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:52:59.733717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 363/717] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:52:59 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:52:59.739870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:59.828727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:52:59.848363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb02fc-2d97-455d-8b78-a0e3437daefc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:52:59.885856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:52:59.886838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:52:59.887359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:52:59.887842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:52:59.888658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:52:59.889153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:52:59.890748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403541d-5b17-47b1-ac06-60d80150e9d4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:52:59.896387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 355/718] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:52:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:52:59.902136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:52:59.931808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:00.010930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1058ba-bdf6-4e03-b700-a442078b6ed3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:00.047957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:00.048954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:00.049397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:00.049882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:00.050764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] RBAC: Adding query filter params (type=network) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:00.051321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:00.052922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a970264-5b76-4dd6-8729-7b522c1eccff None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:00.063219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 364/719] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:52:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:00.343894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02364b-012c-4a15-928a-002ee1093528 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:00.344896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02364b-012c-4a15-928a-002ee1093528 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:00.345398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02364b-012c-4a15-928a-002ee1093528 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:00.347262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 356/720] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:00.353092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346964d3-9b72-4ed9-9f74-4744a5da2174 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:00.353936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346964d3-9b72-4ed9-9f74-4744a5da2174 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:00.354407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346964d3-9b72-4ed9-9f74-4744a5da2174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:00.374081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-346964d3-9b72-4ed9-9f74-4744a5da2174 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:00.375713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-346964d3-9b72-4ed9-9f74-4744a5da2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:00.446147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 365/721] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:53:00.884600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02364b-012c-4a15-928a-002ee1093528 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:00.985872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d785-62df-492e-9c1a-790f029225fa None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:00.986566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d785-62df-492e-9c1a-790f029225fa None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:00.987047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9d785-62df-492e-9c1a-790f029225fa None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:00.988832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 366/722] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:00.994922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:00.995490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:00.995998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.002313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02364b-012c-4a15-928a-002ee1093528 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.029097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:01.032098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:01.067788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.075124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.075124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.075124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.075124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.095151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:01.096034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:01.097082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:01.097975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:01.102574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 357/723] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:01.104419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.119072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.186058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 367/724] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 08:53:01.272764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4d47c-32e2-4168-8ac9-47c85c82fe8f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.326567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.327445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.327869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.328328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.329138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:01.329580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:01.331094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:01.340143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 358/725] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:01.347125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.489083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5304c-63a6-4e90-9495-5bf137f49655 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.495624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.531641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.532573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.533002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.533525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.544495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:01.546351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:01.571774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:01.573019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:01.574114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:01.574114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:01.574114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:01.576424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 368/726] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:01.582208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.590316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.637750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.712750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46970058-127c-49a5-8528-f81a772e7ab1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.760967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c631d-0c36-49ea-9a6c-7cca7f194ea3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.785770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.787480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.787921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.788368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.796929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.797754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.797919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.798197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.798845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:01.799169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:01.800530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:01.801497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8ff79556-a4fd-4e76-acd0-a1f515809252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:01.807405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:01.808388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:01.809257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:01.812032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 359/727] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:01.818249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf2cf-4248-4949-adaa-0867dc666c46 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.818381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 369/728] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:01.818834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.819333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf2cf-4248-4949-adaa-0867dc666c46 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.819829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf2cf-4248-4949-adaa-0867dc666c46 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.821684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 360/729] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:01.822598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbf2cf-4248-4949-adaa-0867dc666c46 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:01.888091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.930051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a14fa-0cd0-4f9a-824e-b29b620b74d9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.931929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbf2cf-4248-4949-adaa-0867dc666c46 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:01.958729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:01.966658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.967288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.967577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.969850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.969961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:01.970062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:01.970062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:01.970220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.970220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.970220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.970452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cce5c-b6e8-420f-a855-d38326bf988d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:01.979205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:01.979515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 361/730] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:01.980754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:01.981039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:01.981366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:01.981528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7eaa1678-bc99-4601-9160-a430bfbd747c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:01.995419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:01.996620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:01.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:01.999690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 370/731] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:02.000773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:02.001219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb496d8-fae7-4767-8858-5940820011cb None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.001632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb496d8-fae7-4767-8858-5940820011cb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.001963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:02.002252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb496d8-fae7-4767-8858-5940820011cb None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.003948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 371/732] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:02.004854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb496d8-fae7-4767-8858-5940820011cb None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.079147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 362/733] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:53:02.088041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.119341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb496d8-fae7-4767-8858-5940820011cb None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.144022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.157047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.157671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.158013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.158229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.170205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.172651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-675ad4f8-0cb4-41a9-b3c8-6d74b0dd1369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:02.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:02.188433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:02.189352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:02.189352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:02.189352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:02.189352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 372/734] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:02.190036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.190310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.190582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.210982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:02.212181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:02.227044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c69e81-03cc-4fbe-b071-daf1150656f9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.250446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.265930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.268733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.268733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.268733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.268733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:02.268733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.269873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.291022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 363/735] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:02.292019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.307631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 373/736] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:53:02.308893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.428573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be49f30-81f1-4156-b155-d2d740f82e73 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.431395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.456768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199fc2a-3225-4ed8-b855-a8d3f1c2497d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.476737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.477737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.478025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.478299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.495957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.497190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.497676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.498149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.499027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:02.499534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.501094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.514015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 364/737] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:02.515078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.560554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.564466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.565932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6620984-7cb7-43d6-9793-e730e0c3b5f1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.624203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc5b95-5413-44b3-9e1f-14671d360bb5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.658361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.658981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.659291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.659571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.667051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 374/738] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:53:02.668399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57885-a88f-430c-b830-ce8f8a8d62ff None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.668601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57885-a88f-430c-b830-ce8f8a8d62ff None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.668867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57885-a88f-430c-b830-ce8f8a8d62ff None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.670264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 375/739] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:02.670768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a57885-a88f-430c-b830-ce8f8a8d62ff None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.672858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:02.675123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:02.678243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:02.685923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 365/740] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:02.687065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.772635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a57885-a88f-430c-b830-ce8f8a8d62ff None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.777222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.780867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52073f-763b-4f77-9e7d-d48770680298 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.807304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.807924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.808202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.808466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.815378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.816149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.816735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.816852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.817014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.817781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.818416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:02.819801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c5c7c-3fb5-44e9-a915-3f972607d68f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:02.829035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:02.830191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 376/741] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:02.831447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:02.895807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:02.927428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 366/742] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:02 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:02.928666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.928958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.929234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:02.934594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df2934-2740-4141-9464-782b653dafb1 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:02.950000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:02.951213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:02.988386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:02.989104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:02.990598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:02.990598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.003481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.005952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:03.022496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:03.023604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:03.024541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:03.024541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:03.024541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:03.024541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 377/743] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:03.025227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.047342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 367/744] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 08:53:03.047342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.155535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.161840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf2a8-a333-4738-9e9b-7f5a9d817b10 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.168990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a00cc-32aa-44fb-ab0f-006d696c8f30 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.177788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.199991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.200647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.200868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.201137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.201789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:03.202086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.203463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.213736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.214353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.214676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.214954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.215725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:03.216102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.217625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.217737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 378/745] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:03.218546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.240882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 368/746] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:03.241738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.342509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320019f-077a-4075-a6af-493d32dcc65c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.356720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a4f2-f208-4910-890a-6708d6635f43 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.378911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.379559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.380674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.380960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.381604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:03.382212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.383633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.401128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 379/747] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:03.401485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.402012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.402525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.402957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.403404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.417167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.419666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.428682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:03.438939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:03.452023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:03.453178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:03.453178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:03.453178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:03.453412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.453500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 369/748] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:03.453500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.511912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69caf00e-fb34-49b7-94c9-ea9f023a6110 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.549755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.550480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.550833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.551207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.566838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28400d18-9d93-4894-a2a1-ce230bf182c1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.606986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.607679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.607963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.608235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.608885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.610264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.626004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.627479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.662510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.695669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 370/749] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:03 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:03.696889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.780014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:03.841357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b0f4a-e2d1-4cf1-8ba0-54d036d5ca9b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:03.859545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 380/750] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 08:53:03.860393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.892093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:03.892321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:03.892524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:03.892723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:03.900104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:03.901572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:03.913891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:03.914837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 371/751] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:03.915831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:03.992462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300f9e-49f5-43b6-8f72-35e60aaa5034 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.037305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.038333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.038813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.039305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.040143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:04.040250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d787-50aa-4eca-b8ed-a8c9ce278abd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.040766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.042499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.044405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.056223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 381/752] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:04.057379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.077021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.088443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.089328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.089616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.089891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.103925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.105338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:04.120526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:04.121544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:04.122548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:04.122548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:04.122548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:04.122548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 372/753] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:04.123310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.192435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b87b6-a382-4ce3-ac38-7f21d002c0b2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.229342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.229975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.230251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.230519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.231365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:04.231552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.232938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.242767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12abac7-e6ef-4224-8704-6114963a2d0b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.247145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 382/754] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:04.247145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.284924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.285863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.286894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.287489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.295748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.297521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:04.310201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:04.311252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:04.312239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:04.314513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 373/755] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:04.315622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.323792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.348175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.387479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350608-bbc1-481f-8ed8-56b68f6d99e7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.426527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.428086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.428945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.429470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.430312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:04.430814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.432657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.435339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527890a-83ce-4905-a72d-b23c56c3bc6b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.447090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 383/756] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:04.448490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.480313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.480921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.481210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.481488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.482483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=0c95310969054beab60e99fd71a34aee) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:04.482796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.484934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.495191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 374/757] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/role_assignments?user.id=0c95310969054beab60e99fd71a34aee&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:04.496405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.630148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b01dc-77c6-42dd-b11e-0c53eda3cd72 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.668635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.694041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:04.696858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.697638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.698152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.698569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.699499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:04.699954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.701725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.706998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940c1-511b-455e-84d4-23b7b72785ba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.721022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 384/758] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:04.727515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.748386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.749132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.750146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.750551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.763158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.763158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:04.773549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:04.774685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:04.775853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:04.776883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 375/759] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:04.788211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.862191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053216ab-f19c-4b4c-b033-de095351da2c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:04.908527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:04.909173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.909510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.909789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.932276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:04.933736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-439c7a51-fd09-446f-982c-a83a9197d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:04.969651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:04.970902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:04.972158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:04.972158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:04.972158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:04.972158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 385/760] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:04.972441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49504729-d1ca-4b2b-a6da-03bf98dd8f53 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:04.972702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49504729-d1ca-4b2b-a6da-03bf98dd8f53 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:04.972974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49504729-d1ca-4b2b-a6da-03bf98dd8f53 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:04.978936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 386/761] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:04.978936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49504729-d1ca-4b2b-a6da-03bf98dd8f53 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:04.994507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faf031-4657-4ee9-a7d1-f05f3d5dbc4d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.011472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:05.031754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.032373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.032655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.032937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.034017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:05.035213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.037535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0f22a-5a52-4467-aaaf-7f5e46f9e05b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.053914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 376/762] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:05.055939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.056390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.056892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.087700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:05.089106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:05.093132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:05.100637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49504729-d1ca-4b2b-a6da-03bf98dd8f53 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.137349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.138178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.138595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.139036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.139901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:05.140341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.141900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.155129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 387/763] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:05.156202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.183156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 377/764] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:05.183156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.297763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab9ce6-ef58-4dd1-8de9-242ef951bd50 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.310844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:05.338107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.338923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.339290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.339509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.346731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8da0-a834-481c-a2f5-a86e0b671bb4 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.347292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.351288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.352205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:05.363629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:05.372138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:05.373270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 378/765] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:05.376705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.417672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.418368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.418735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.419115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.419936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=e9d0bba06c9c4425a0e96641dab46c19) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:05.420818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.423190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.433217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 388/766] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:05 2022] GET /identity/v3/role_assignments?user.id=e9d0bba06c9c4425a0e96641dab46c19&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:05.534114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a42f8-d40a-42ec-8de2-b1b5e6920493 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.588023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.588849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.589229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.589636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.590442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:05.590848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.592423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.613398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 379/767] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:05.619371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.622521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.705600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:05.751403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62fa53-ad2f-44c6-857f-6203762557d0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.776851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03322581-ca37-4a13-a997-0afe47512387 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.793715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.794604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.795156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.795579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.796441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.797965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:05.811991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:05.813210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:05.814268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:05.816818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:05.817144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 380/768] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 08:53:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:05.822632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.822632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.822632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.823788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=d132894fe36047418e5cd8bebe04c387, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:05.824116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:05.825435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427801cc-4947-471e-a79f-c8e50a640907 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:05.923319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 389/769] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:05 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:53:05.927577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cdbf-b529-49e9-a4a9-1b17cb0a4fa2 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.928117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cdbf-b529-49e9-a4a9-1b17cb0a4fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.928588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162cdbf-b529-49e9-a4a9-1b17cb0a4fa2 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.932786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 390/770] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:05.939599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:05.940427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:05.940999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:05.962779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5c56-b2b6-44a0-9432-b0fa50ccc0f9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:05.983668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:05.985286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:05.990616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.023077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.045946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.046985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.048772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.049210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.050811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:06.051697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.053733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.059400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 381/771] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:53:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:06.065249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:06.135273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 391/772] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:06.180132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73058c1e-2380-44e2-82a9-2b7a26bba7fc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:06.216947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.217484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.217767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.218042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.218663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:06.218964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.220375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.225987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 382/773] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:06.231825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:06.352078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b811a-1c8d-493d-9d2c-524015702bb3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:06.388461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.389224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.389597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.389961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.390746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] RBAC: Adding query filter params (interface=public, service_id=da5d532bd18e49de998f5dc9b8c6cb70, region_id=RegionOne) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:06.391167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.392607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.400626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 392/774] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/endpoints?service_id=da5d532bd18e49de998f5dc9b8c6cb70&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:06.405727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:06.486882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:06.490368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.492359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.569834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a43bf-03cc-416e-b035-63e17a20af0a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:06.628213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.628818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.629115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.629384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.630528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.631933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.638308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41117-b1dd-489f-b6ba-58bd62059af0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:06.644245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 383/775] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/services => generated 772 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:06.688939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.689674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.689959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.690232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] PATH_INFO: `/v3/services/network` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.690827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.692826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-84305684-1f6e-40d2-8ec6-5a59075ad1af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:06.708039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:06.709626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:06.710494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:06.711270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 393/776] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:06.717817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:06.728010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d9dc-4176-4382-a791-e6cea3a25f6e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.729323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d9dc-4176-4382-a791-e6cea3a25f6e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.729830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d9dc-4176-4382-a791-e6cea3a25f6e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.731820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 394/777] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:06.738558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357766b8-a627-486f-8327-f617766942b2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.739185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357766b8-a627-486f-8327-f617766942b2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.740204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357766b8-a627-486f-8327-f617766942b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.762230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-357766b8-a627-486f-8327-f617766942b2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:06.764027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-357766b8-a627-486f-8327-f617766942b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:06.854727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 395/778] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:06.858654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.862532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63ff4-7e80-4587-9ef1-5e6ec09e0316 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:06.881661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:06.900597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:06.901209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:06.901606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:06.901884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:06.902572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] RBAC: Adding query filter params (name=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:06.902882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:06.904304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:06.909337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 384/779] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:06.915329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357766b8-a627-486f-8327-f617766942b2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.022659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357766b8-a627-486f-8327-f617766942b2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.058168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:07.058776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.059058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.059367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.063215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] RBAC: Adding query filter params (type=network) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:07.063215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:07.063215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:07.066890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 396/780] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:07.076415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.203841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843feb57-98ec-42c4-8d33-7f90a6bcb592 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.251942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:07.252555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.252838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.253114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.253809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] RBAC: Adding query filter params (interface=public, service_id=886c9ee115964677a9bd3d315caa3dd9, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:07.254118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:07.255599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:07.269665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 385/781] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:07 2022] GET /identity/v3/endpoints?service_id=886c9ee115964677a9bd3d315caa3dd9&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:07.275746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.341185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:07.358810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.406878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7407227-94c7-49db-9712-375e0daa7385 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.451503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:07.485255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a20b6-290a-4bf5-9ba3-838b1bae97a8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.515826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:07.516456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.516739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.517012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.517613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:07.518995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:07.522906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:07.523755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.524532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.524984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.538283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 397/782] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:07 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:07.544036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:07.549747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:07.550698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:07.551591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:07.563195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 386/783] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:07.566860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.724952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba36e2-a3d2-4c61-baa8-da7a2352d057 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.751231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:07.762471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:07.770062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a3f74-0d2e-4d3d-bd94-0b44e595cd30 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:07.776267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 398/784] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:07.781504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:07.875188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c09-2325-4dc0-a8dc-3ccf6b2b2811 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.876118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c09-2325-4dc0-a8dc-3ccf6b2b2811 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.876586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc49c09-2325-4dc0-a8dc-3ccf6b2b2811 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.878369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 399/785] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:07.885152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:07.885691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:07.886160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:07.915245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:07.917501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:07.944715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f36994-6ec6-4da7-b694-95e9c73e4aa1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:07.954770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:08.006416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.007050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.007352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.007623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.018920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:08.020496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:08.036324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:08.037497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:08.039002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:08.039002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:08.039002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:08.039002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 387/786] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:08.043019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.054899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 400/787] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:08.198819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee649-bfe9-40a4-a34e-b49262939c68 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:08.248731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.249347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.249631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.249909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.250556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:08.250862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:08.252351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30cd4a-be22-47e1-a379-d16162010862 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:08.270213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 388/788] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:08.276056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.338726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355014be-78e6-4257-b361-25f2f443b144 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.339027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355014be-78e6-4257-b361-25f2f443b144 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.339314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355014be-78e6-4257-b361-25f2f443b144 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.340806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 389/789] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:08.349278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.349558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.349856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.392108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:08.393608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:08.393719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad65fe-1010-4ead-a6ac-140389956859 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:08.408461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:08.416186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:08.439094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.440094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.440961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.441433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.453260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:08.455569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:08.475195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 390/790] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:08.475195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:08.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:08.478154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:08.479122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:08.482100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 401/791] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:08.488605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.660540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed20d-ac78-44e8-b81f-c7512e5b9ddd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:08.675462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40026-445f-40ca-a52d-cc9f504db359 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:08.709412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.711754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.712242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.713472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.739825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:08.751129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.751129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.751129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.751129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.751488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:08.751488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:08.751488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:08.755875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:08.756837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:08.757687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:08.760329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 391/792] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:08.766170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.768800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 402/793] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:08.782726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:08.783646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.878069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:08.884920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4c25d-9d27-44de-a67b-eb425caf33f2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:08.928159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:08.929859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:08.930327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:08.930781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:08.931672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:08.932178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:08.934170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:08.944858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 392/794] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:08.949919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:08.970878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48379d79-f4fe-42f1-bae3-fcbf40a9240e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.022235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.023661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.024702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.025479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.038575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.041925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.043594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:09.059602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.060585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:09.061992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:09.061992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.061992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:09.065482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 403/795] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:09.066326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.076255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d172d0-7fe3-40a4-a374-2424d3c9cf79 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.113155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.113772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.114054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.115415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.127889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b639a135-55ae-4a4b-b682-8c1b7959c1c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:09.135329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.136306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:09.137171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:09.138535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 393/796] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.139776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16066d1c-6c7c-4010-8509-f7b82d3737b7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.140060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16066d1c-6c7c-4010-8509-f7b82d3737b7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.140336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16066d1c-6c7c-4010-8509-f7b82d3737b7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.142878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 394/797] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:09.142878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16066d1c-6c7c-4010-8509-f7b82d3737b7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.165503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.183522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35b7b1-0cd2-40d7-95d4-c063d502b8b3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.219492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.220201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.220380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.220641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.230953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.232348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:09.247838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.248804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:09.249752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:09.249752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.249752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:09.249752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 404/798] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.249752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.262714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16066d1c-6c7c-4010-8509-f7b82d3737b7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.290523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.299059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.304401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce244e9e-339d-4d5d-8b1b-3b8f7afe14da None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.319878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 395/799] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:09.319878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.319878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.319878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.337028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:09.338224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:09.361902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614f921-a140-4b4e-921a-01010f93a10c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.400525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.401579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.401928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.402270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.403004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:09.403441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.404753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.410242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 396/800] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:09.410976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.411510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.424769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 405/801] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:09.426279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.542059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa2b02-c226-424f-9a0a-18f2f7fe99a7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.547073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07315b9-ac6d-4838-ae6d-7ae756206972 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.566497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.587917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.589058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.589543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.590017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.590862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:09.591485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.593094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.593950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.594571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.594845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.595134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.604923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.606606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.612213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 397/802] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:09.613429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:09.620906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.621838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:09.622750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 406/803] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.623280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.686872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.729117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db76b-824c-4a0c-ab27-320741379889 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.733231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215e6bf-3c1c-4937-b472-fb6b508b8841 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.766069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.766695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.766977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.767327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.768551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.769195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.769460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.769730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.778951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.779120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.780410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.780536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:09.793320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.794280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 407/804] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.795886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:09.798412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.799924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:09.800864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:09.800864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:09.800864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:09.800864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 398/805] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.801093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.834131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.906042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37fa3-a34f-4fae-bf83-13045c30abd1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.907435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7c293-89aa-424a-86c3-599fb124f772 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:09.927937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:09.939732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.940348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.940628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.940895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.941187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:09.941457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.941787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:09.942067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:09.942429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:09.942851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.943242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:09.943544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:09.944977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:09.951832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:09.952872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:09.953751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 399/806] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:09.954754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:09.965482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 408/807] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:09.965482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.049112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.055686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c9c41-81b3-4156-bc70-60dd3e7f7015 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.060651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14671cc-1e5c-4561-ac34-e4f003868aa6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.089069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.089675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.089956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.090229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.090949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.091286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.092719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.094096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.094696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.094977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.095286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.095919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.096220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.097559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f45cc-9d67-499b-ac9a-32d25316f25c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.109331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 409/808] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.114523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 400/809] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.114668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c71d7-e7a7-433d-b41e-d4a5939b6577 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.114668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c71d7-e7a7-433d-b41e-d4a5939b6577 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.114806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.114901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c71d7-e7a7-433d-b41e-d4a5939b6577 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.115353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 410/810] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:10.116005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c71d7-e7a7-433d-b41e-d4a5939b6577 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.158312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.214220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51910c97-e85f-4026-8faf-63db89f71923 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.216513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c71d7-e7a7-433d-b41e-d4a5939b6577 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.248002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.248600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.248880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.249155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.249777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.250078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.251506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.255019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.255839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 401/811] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.255980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.256204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.256475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/0c95310969054beab60e99fd71a34aee/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.256598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.258374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=0c95310969054beab60e99fd71a34aee, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.259829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.274576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.329237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 411/812] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:10 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/0c95310969054beab60e99fd71a34aee/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:53:10.329805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.359070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cdf3c-e07b-454d-b7ce-8e87ca3c96e4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.398948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.399624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.399903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.400175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.408151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.409640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2ffdfff7-00a8-4757-96ba-71bbef9e4caa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:10.422623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:10.423851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:10.424865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:10.428840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 402/813] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:10.430240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.430530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.430806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.441048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.445371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856acd4-3123-495a-803f-7f48b1900cc0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.449950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:10.451215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:10.491514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.492539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.492915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.493090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.504111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.505979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:10.522143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:10.523231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:10.524165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:10.524165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:10.524165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:10.524165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 412/814] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:10.524165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.538698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 403/815] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:10.540456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.562244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.639287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25476767-ddd4-403e-97ea-4ea181561669 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.679651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7c3d-f353-4ba5-8b51-c2bfa7dc5eb6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.683806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.684409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.684684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.684947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.685565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.685859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.687235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.695515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 413/816] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.696294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.719446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.724909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.725876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.726355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.726938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.728050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.728518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.730562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.750626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 404/817] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.752067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.805337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbf4c9-15ce-4cb2-9430-82d8ba0d3575 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.824236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.840171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.840772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.841047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.841547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.841934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:10.842230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.843628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.855154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 414/818] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:10.855976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:10.871262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c2ec6-b0b4-4221-be85-7a7b5431619b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.907937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.908501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.908726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.908995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:10.909648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:10.911004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:10.939430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:10.961911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849c1cf-fd6b-42ae-a167-32cce9311b3a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:10.994541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:10.995162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:10.995437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:10.995703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.013546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 405/819] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:11.014551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.088139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.090175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.172690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9a35-ca2d-4e39-ba62-cfcbfe849d94 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.182416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.194638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.213590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 415/820] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:11.214456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.217860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.218672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.219162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.219958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] PATH_INFO: `/v3/services/network` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.220561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.221950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:11.232017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:11.233561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:11.234439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 406/821] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:11.235330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.335795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1530f6a-6c5f-4e7b-9f83-92b83cae5371 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.372808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7954d0c-2e16-4402-b42b-b0ee1d33c814 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.378198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.378899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.379215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.379485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.390449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.392379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:11.407736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:11.408705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:11.409628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 416/822] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:11.410134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.421201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.421841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.422756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.422976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.438060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.439620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.452962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:11.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:11.456502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:11.457608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:11.457608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:11.457608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:11.461707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 407/823] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:11.462684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.503490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.528910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aa0ce-b340-407d-97ff-2b9fb5530b07 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.565974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.566581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.566856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.567146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.567964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] RBAC: Adding query filter params (name=network) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:11.568507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.569877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.575908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 417/824] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:11.576759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.581301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff87-2a50-446f-99f8-8d9e66bd0bb1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.619248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.620211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.620719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.621279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.622161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:11.622661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.624421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.637915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 408/825] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:11.639012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.689966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.694223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16254ce-7313-4fc8-affa-e94c6f8e9579 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.732032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.732639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.732918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.733187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.734132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:11.734319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.735755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.738062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:11.748070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccd3da-fe46-4ec9-b801-33d7730d2ccd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.750656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 418/826] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:11.751801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.786423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.787494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.788136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.788249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.789037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] RBAC: Adding query filter params (type=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:11.789420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.790807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.796506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 409/827] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:11.797374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.865394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d698e7-97b6-4fac-8ba0-744623e34941 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.902209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.903323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.903785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.904287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/e9d0bba06c9c4425a0e96641dab46c19/roles/304d887d3553492bbb31f573a7952646` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.906282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b72dc-7634-4762-84c9-c661d430315b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:11.921520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=e9d0bba06c9c4425a0e96641dab46c19, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.923428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.950364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:11.950364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:11.950364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:11.950364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:11.951857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:11.953399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:11.965283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:11.966194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:11.967137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 410/828] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:11.967750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:11.989911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:12.011143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 419/829] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:11 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/e9d0bba06c9c4425a0e96641dab46c19/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:53:12.011143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:12.033546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:12.178881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988c777-939a-44f2-a628-739d8111cb95 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:12.197683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f89df-8898-4b27-b55b-a8c7e5f694a8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:12.246680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:12.247245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:12.247522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:12.247793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:12.248534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:12.248838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:12.250380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:12.264730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 420/830] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:12.271757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:12.352327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:12.352933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:12.353207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:12.353470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:12.354129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:12.355532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:12.369374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:12.401106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e03d43-b5e8-41ce-b947-f678705cc34f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:12.433889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:12.434520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:12.434787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:12.435051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:12.435787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:12.436085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:12.437487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932d891-8d36-4875-844b-048ab6a92011 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:12.443780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 421/831] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:12 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:12.451242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97599c8-f231-4799-a5fe-4d1c83b97c12 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:12.451242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97599c8-f231-4799-a5fe-4d1c83b97c12 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:12.451242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97599c8-f231-4799-a5fe-4d1c83b97c12 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:12.451242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 422/832] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:12.457635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:12.458114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:12.458480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:12.482366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 411/833] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:12.491755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:12.493374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:12.612366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 423/834] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:12.906542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:12.981948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.016499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2edc8-9245-4bab-8a54-d1a3d7f64a4e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.057517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.058421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.058845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.059281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.069998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f21451df-db5f-43a2-a3e8-b0d1d31e8c7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:13.075377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:13.076500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:13.077539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:13.079829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 412/835] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:13.087156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.137445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34fa9-c9b7-4b54-b206-24123d2cea6b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.137724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34fa9-c9b7-4b54-b206-24123d2cea6b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.137993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34fa9-c9b7-4b54-b206-24123d2cea6b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.139496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 413/836] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:13.144262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.144540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.144818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.160245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.164452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:13.165650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:13.205228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f8f2c-62f1-4964-8a9d-80848262aefa None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.240990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 414/837] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:13.248136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.248950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.249373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.249785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.250585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:13.251027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.252615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.265662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 424/838] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:13.271808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.337483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.387510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6804-0486-4424-a354-281e6e7fdad8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.422103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.422655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.422929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.423229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.434578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.436049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:13.450372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:13.451339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:13.452277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:13.452277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:13.452277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:13.452277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 415/839] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:13.460560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.542444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.570733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f3586-e67d-4bce-a8ed-129fc6579815 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.595078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.607049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.614360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.623515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 425/840] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:13.629097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.660764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcee32-4f01-41c2-9a9c-27b48536db3f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.675699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.697261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.698157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.698578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.698989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.699725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.701243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:13.711091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:13.712153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:13.713089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:13.720758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 416/841] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:13.721867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.735806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c80c45-6dca-4801-9030-58e6d6b40376 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.774486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.775129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.775425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.775693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.783629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.785320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:13.798170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:13.799217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:13.800223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 426/842] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:13.805609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.832534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70070f-fcce-44b0-b7b3-3047a6622061 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.868109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.872445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.873355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.873839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.874369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.875268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:13.875784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.877383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.887692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 417/843] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:13.888808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:13.913266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e321df7-69f9-4794-ae31-720957c41373 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:13.936576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:13.950354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:13.951938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:13.951938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:13.951938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:13.952291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:13.952590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:13.954015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:13.976639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 427/844] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:13.977534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.001379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389454-c8cc-45de-acd6-a6b60a55133a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.039739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.040735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.041292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.041855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.042686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:14.043222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.044837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa16c0-a0a1-4b9b-8111-115ee0b907c2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.055301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 418/845] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:14.082229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:14.086193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3332ae-4f75-4c66-802c-e0156a385602 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.126721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.127434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.127672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.127929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.181750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8b21b-1f61-4cf2-b913-93236f3a01b7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.182066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8b21b-1f61-4cf2-b913-93236f3a01b7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.182310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8b21b-1f61-4cf2-b913-93236f3a01b7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.188065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 419/846] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:14.189784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.190174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.190456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.205010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.206638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.211579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:14.213066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:14.315140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 420/847] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:14.332071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 428/848] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:14.336891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:14.338147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.378009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:14.450245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a9c6b-2871-4134-8d90-2adc7ee3d5e7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.486134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.486761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.487044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.487346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.497970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.499454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:14.514380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:14.515391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:14.516331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:14.516331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:14.516331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:14.516331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 421/849] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:14.523022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.632042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54674af-d71c-48cb-ac3f-a7a6a176ae9b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.636334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.670912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.671877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.672432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.672898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.673885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:14.674381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.676046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.694549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 429/850] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:14.695917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.744878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:14.757454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3778da-5180-4697-b856-a5da989849d7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.761556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:14.794157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.795083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.795384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.795655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.803679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa55c-506a-4eef-955a-87f5dd0779d2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.806350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b228da09-3517-4257-8518-3b8f4170f8d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:14.812372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:14.813319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:14.814187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 422/851] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:14.814981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea3fd5-1f22-45fa-b1d6-944deb044f57 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.814981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea3fd5-1f22-45fa-b1d6-944deb044f57 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.815157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea3fd5-1f22-45fa-b1d6-944deb044f57 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.816422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 423/852] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:14.821517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea3fd5-1f22-45fa-b1d6-944deb044f57 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.845946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.846570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.846851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.847141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.853849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.855321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-b12fe2ec-581b-49c4-917a-b837a237271b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:14.867860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:14.868782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:14.869706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:14.874116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 430/853] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:14.875190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.875472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.875749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.896088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:14.897295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:14.934969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea3fd5-1f22-45fa-b1d6-944deb044f57 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:14.967930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 431/854] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:14.969077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:14.979933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:14.981045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:14.981326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:14.981596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:14.982255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:14.982555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:14.983950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:14.998117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 424/855] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:14.999198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.013267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.039549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.094070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbea54-1d11-477c-92b2-d2a9255977bc None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.135266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.135995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.136220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.136494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.137221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.137524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.139282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.142001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598fb4-1092-474e-bd22-fbd067fb1c36 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.151901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 432/856] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:15.157865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.186211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.187033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.187431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.187797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.202192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.204726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:15.223555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:15.225949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:15.227290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:15.227290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:15.227290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:15.227290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 425/857] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:15.233064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.284935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4f4b3-c057-4fb3-8ff7-e21797a3d340 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.296562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.297993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.321558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.322163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.322525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.322771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.323514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=0c95310969054beab60e99fd71a34aee) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.323819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.325295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.331272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 433/858] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/role_assignments?user.id=0c95310969054beab60e99fd71a34aee&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:15.332362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.346488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2c0-a852-4b0f-a809-c5c1ec4b7e1f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.391604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.392028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.392315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.392593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.393249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.393555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.395405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.416383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 426/859] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:15.417731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.469089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab860c-dc21-4d75-9e98-311fed45895e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.511652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.512294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.513289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.513566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.514161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.516935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:15.529115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:15.530115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:15.531033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 434/860] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:15.537246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.552539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd3390-7ea1-431b-a241-782a00a34577 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.583731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.585053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.600797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.601435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.601874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.602158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.612376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.615425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:15.632879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:15.634005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:15.634953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 427/861] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:15.639946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.657853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a78-3be8-4120-a67d-a08e942063a1 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.703590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.704525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.705086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.705369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.706001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.706469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.708764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.719214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 435/862] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:15.721906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.754713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a6bc-7952-40fe-a744-1cf1182ef980 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.792957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719915b0-f0a8-404e-8521-389604534148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.793536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719915b0-f0a8-404e-8521-389604534148 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.793819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719915b0-f0a8-404e-8521-389604534148 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.794091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719915b0-f0a8-404e-8521-389604534148 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.794816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719915b0-f0a8-404e-8521-389604534148 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.795132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719915b0-f0a8-404e-8521-389604534148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.796591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719915b0-f0a8-404e-8521-389604534148 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.816743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 428/863] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:15.823117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719915b0-f0a8-404e-8521-389604534148 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:15.850267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900446f-71df-421e-adbb-69d4549b3f57 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.852056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.867678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:15.899288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.900943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.901601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.902188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:15.903312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:15.903772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:15.905225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4742156-08b4-4ae3-a0f5-71f68bb5e3ca None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:15.915150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 436/864] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:15.946977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719915b0-f0a8-404e-8521-389604534148 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:15.986832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:15.987492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:15.987750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:15.988030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.037529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02307be-5cd1-4b45-a9af-03f8402338c7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.037865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02307be-5cd1-4b45-a9af-03f8402338c7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.038063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02307be-5cd1-4b45-a9af-03f8402338c7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.039769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 437/865] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:16.045392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.045653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.045858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.065831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:16.067273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:16.072130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.073626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.125565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:16.153008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 438/866] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:16.194201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 429/867] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:16.200155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.300928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:16.307206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db9dbd-54b4-4f47-b643-3e4d6477216e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:16.342997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:16.343656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.343935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.344201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.355342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.356786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:16.373015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:16.374043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:16.375599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:16.375599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:16.375599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:16.375599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 439/868] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:16.380060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.422893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.497132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9684076-005a-4c19-add0-77d4697c2441 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:16.537197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:16.537908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.538248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.538586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.538929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996a6f-ff52-41d8-8014-0c8942dfb31e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:16.539435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:16.539801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.541427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.562442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 430/869] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:16.562746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:16.570418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.585272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:16.586303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.586596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.586872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.587564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.589538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.671929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 440/870] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:16 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:16.691084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54c882-6d2c-4288-b4fa-207a2648df80 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:16.738911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:16.739619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.739900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.740176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.748264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.749752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:16.762540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:16.763549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:16.764492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 431/871] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:16.769760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.784187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:16.876025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069263-7b0e-4af0-bc7f-e5719674d573 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:16.912658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:16.913523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:16.914490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:16.914760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:16.915499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:16.915804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:16.917243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd16e6-438b-402c-89af-985e542a279b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:16.929986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 441/872] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:16.935958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:16.997549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:17.045480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fc2d2-ebed-45f1-a1b7-f8bdc502dd04 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:17.082097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:17.082716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.083034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.084035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.084890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=e9d0bba06c9c4425a0e96641dab46c19) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:17.085205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:17.086768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438e490-0ff6-43e8-aa21-18697f3b3ef4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:17.094082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 432/873] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:16 2022] GET /identity/v3/role_assignments?user.id=e9d0bba06c9c4425a0e96641dab46c19&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:17.375370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4a2c-96b8-4d7d-960d-c66879532d79 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.375916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4a2c-96b8-4d7d-960d-c66879532d79 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.375916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4a2c-96b8-4d7d-960d-c66879532d79 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.377471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 442/874] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:17.383090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.383404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.383675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.402327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:17.403539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:17.472746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 433/875] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:17.746928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc4a2c-96b8-4d7d-960d-c66879532d79 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:17.817238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:17.855732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc4a2c-96b8-4d7d-960d-c66879532d79 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:17.893438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:17.894407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.894903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.895422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.896720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:17.898328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-88ff8d25-338e-4379-83ea-b2f7949a5f3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:17.907588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:17.908607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:17.909469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:17.912188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 443/876] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:17.920487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:17.953293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06306-00f6-4da6-9596-413ee7081dd0 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.953631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06306-00f6-4da6-9596-413ee7081dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.953825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06306-00f6-4da6-9596-413ee7081dd0 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.955404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 444/877] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:17.961142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:17.961662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:17.962085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:17.973335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:17.980957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:17.982349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:18.037795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc07327-f695-468a-914b-778f0abb5720 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:18.067222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 445/878] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:18.074413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:18.075018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:18.075329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:18.075608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:18.076235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:18.076545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:18.077913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:18.082880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 434/879] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:18.088717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:18.152067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:18.193094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aa3c1-c380-4867-9099-9cee81bf15b1 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:18.229416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:18.230030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:18.230308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:18.230578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:18.231233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:18.231538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:18.232898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c8f47-66bb-4c10-81e1-a4d15ecd6cc1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:18.237904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 446/880] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:18.357968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:18.467605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328478fd-cb87-431a-9464-855ba67031b8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:18.483344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:18.504982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:18.506086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:18.506368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:18.506645 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:18.507371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:18.508840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54426b2d-c79b-4c9a-bacd-1b9904a44e24 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:18.621263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 435/881] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:18 2022] POST /identity/v3/services => generated 269 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:18.749990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45571651-b4da-4e90-9a5e-ce094c7482d4 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:18.750564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45571651-b4da-4e90-9a5e-ce094c7482d4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:18.750564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45571651-b4da-4e90-9a5e-ce094c7482d4 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:18.752233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 447/882] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:18.758494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99b149-3058-4fbf-b0e6-472afde4c048 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:18.758788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99b149-3058-4fbf-b0e6-472afde4c048 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:18.759069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99b149-3058-4fbf-b0e6-472afde4c048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:18.777852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1e99b149-3058-4fbf-b0e6-472afde4c048 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:18.779057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1e99b149-3058-4fbf-b0e6-472afde4c048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:18.861972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 436/883] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:19.131445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45571651-b4da-4e90-9a5e-ce094c7482d4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.159540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7080-d93c-43cb-b78e-1920b0dc0beb None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.160525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7080-d93c-43cb-b78e-1920b0dc0beb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.161007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7080-d93c-43cb-b78e-1920b0dc0beb None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.162997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 437/884] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:19.168186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.168714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.169199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.188212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:19.189716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:19.240539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45571651-b4da-4e90-9a5e-ce094c7482d4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.267226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 438/885] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:19.268027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:19.277480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.278167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.278448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.278721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.279376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.280790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:19.291202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:19.292328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:19.293261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:19.297318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 448/886] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:19.298679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.337102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:19.411955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1571f-cf44-4e4d-bea6-fdb234f89d10 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.448239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.448839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.449120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.449393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.450011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:19.450313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.451716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.456690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 439/887] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:19.464466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.569250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff9628-0bde-487f-8ca7-5e95bef0c86d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.590203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.604796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.605402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.605682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.605956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.606634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:19.606943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.608386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.619143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 449/888] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:19.620390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.682436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:19.704776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:19.712464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e63b-d52f-47dd-bf00-0c5b58bedd15 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.731310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ebed4-7500-44be-89b2-e80766b7b672 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.760870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.762056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.762907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.763501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.767568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.768258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.768606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.768948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.769715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] RBAC: Adding query filter params (interface=public, service_id=2dd2f3d42ecf4c71bec68581d12f9428, region_id=RegionOne) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:19.770108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.772558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.778298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:19.781057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 450/889] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/endpoints?service_id=2dd2f3d42ecf4c71bec68581d12f9428&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:19.788486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:19.788776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:19.789747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:19.790620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 440/890] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:19.796267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:19.901378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e496e-9c7f-4dfa-9918-b831228fc700 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.904218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9e4f-9798-46a9-97bf-2219d2b58e39 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:19.922618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:19.937821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.938888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.939422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.939953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.940939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:19.941508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.941981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:19.943329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:19.943740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:19.944197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:19.944523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.944927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:19.946346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:19.955154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 451/892] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 08:53:19.957928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 441/892] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:19.959366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.091442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f14a11-819c-4943-91d8-592bb03cf0d7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.130011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.130923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.131388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.131821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.143674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.145348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-ee8a6d43-88ad-4d07-b52f-3c9d71c17f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:20.160711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:20.162555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:20.163687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:20.163687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:20.163687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:20.165439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 452/893] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:20.175143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.191255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:20.234984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16955f-5240-46d7-9faf-06843327dceb None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.235326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16955f-5240-46d7-9faf-06843327dceb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.235568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16955f-5240-46d7-9faf-06843327dceb None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.237153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 453/894] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:20.243313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.243845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.244311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.263939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:20.265587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:20.281822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb692353-0440-4b19-b4ba-cc9d0e11a5d5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.296922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:20.318290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.318900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.319214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.319479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.320122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:20.320423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.321861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.339150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 442/895] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:20.341566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.359185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 454/896] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:20.432868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:20.457913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ec842-4975-46f1-8fcf-8877b435dac1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.500681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.501313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.501602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.501878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.509228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.510691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:20.523272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:20.524238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:20.525227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 443/897] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:20.531688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.649224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3701a2-055b-44d1-9392-51d3bbca104e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.658717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.687292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.688349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.688918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.689402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.690321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:20.690828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.693302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b1cd-da89-4803-a3de-5eb233b245fe None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.712313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 455/898] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:20.718293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c42-385e-4a6a-8d97-66af491fa345 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.718573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c42-385e-4a6a-8d97-66af491fa345 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.718844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6c42-385e-4a6a-8d97-66af491fa345 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.720236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 456/899] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:20.720784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb6c42-385e-4a6a-8d97-66af491fa345 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:20.746197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:20.767328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ab5d-fae5-45d0-aba3-a6be29e3100b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.802919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.803535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.803820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.804088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.804735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.806100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5cef8-9153-409a-b9ed-4e7ee3a216a1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.810623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:20.828482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb6c42-385e-4a6a-8d97-66af491fa345 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:20.863420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:20.864021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.864918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.865173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.888026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 444/900] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:20 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:20.898982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:20.899512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:20.903622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:20.930635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:20.932468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:20.942271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:20.943978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:20.976077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.026737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 445/901] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:21.071234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 457/902] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:20 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:21.076658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.183534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96561ec-878b-47a9-b9eb-db866027a14f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.210253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.222273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.222901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.223211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.223487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.234535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.235260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:21.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:21.251519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:21.252446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:21.252446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:21.252446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:21.252446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 446/903] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:21.259118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.319278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.367694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cbb25-f8b4-4c64-a919-da88919f3d51 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.395208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.407703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.408659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.409177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.409643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.410537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:21.411031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.412989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.425461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 458/904] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:21.432430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.441912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5a36a-80a1-4294-938d-36b87a6e450f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.486670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.487277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.487562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.487842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.488529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.490483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:21.500783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:21.501773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:21.502674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 447/905] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:21.515129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.557117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.561849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e3de-f657-4a9d-87d3-cb7b183a1bae None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.612098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.612788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.613075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.613351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.623979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.626145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.632095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c2bc8-90c7-446d-96db-c9b722457ebf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:21.641148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:21.642100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:21.643037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:21.647272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 459/906] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:21.649704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.675224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.675827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.676104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.676373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.677650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:21.677957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.679486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.684306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 448/907] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:21.689708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.711198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.789086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781596f-3e52-4234-95b7-236f6524b249 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.796790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd62a6a-4123-4781-95e9-4830f3fc6682 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.798617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.826364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.827000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.828125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.828392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.829169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:21.829718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.830881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.838919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:21.839914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:21.840421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:21.840920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:21.841842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:21.842359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:21.843964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:21.850186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 460/908] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:21.852143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.858453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 449/909] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:21.861007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:21.982839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305ba36-1d30-46c6-9199-7c343f4f5b8c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:21.984712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:21.986048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a551762-f749-4cf0-9b7a-e95f7252e8cc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:22.019580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.020437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.021299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.021682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.022197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] RBAC: Adding query filter params (role.id=f8fd612f0e494586a34c8c2079efa989, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=e9d0bba06c9c4425a0e96641dab46c19) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:22.022495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.023140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.023750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.024004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3fe4d-9deb-421c-aee0-0f29eabdf30c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.024115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.024290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.024983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] RBAC: Adding query filter params (interface=public, service_id=ec086f6b1f034d0ca88fb6fa274011fa, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:22.025293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.026658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.037931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 461/910] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/role_assignments?user.id=e9d0bba06c9c4425a0e96641dab46c19&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=f8fd612f0e494586a34c8c2079efa989 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:22.037931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9bf45-16e1-4b16-aeff-8f6c028a846b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.037931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9bf45-16e1-4b16-aeff-8f6c028a846b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.037931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9bf45-16e1-4b16-aeff-8f6c028a846b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.037931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 462/911] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:22.039589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.039816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.040088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.041002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 450/912] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:21 2022] GET /identity/v3/endpoints?service_id=ec086f6b1f034d0ca88fb6fa274011fa&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:22.048343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:22.063490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:22.064749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:22.088058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:22.166140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 463/913] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:22.177651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800f8d8-b2fb-4bb2-917e-3cf310331ee1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:22.214567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.215231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.215467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.215747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.216346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.217706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.223929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 451/914] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:22 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 08:53:22.436920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:22.536957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:22.543037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc040a-2fe8-44f0-939e-0da1bc9ddd9c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:22.589694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.590321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.590594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.590857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.591466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.592821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:22.602287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:22.603324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:22.604209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 464/915] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:22.609513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:22.706144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:22.713956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183872b1-1696-405e-a28c-49ab27086e73 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:22.748790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.749392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.749670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.749940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.750560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:22.750859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.752258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.757919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 452/916] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:22.763485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:22.864892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25819ce-dce2-43d5-acc1-1173798278ae None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:22.899277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:22.899963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.900238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.900503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.901128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:22.901415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:22.902769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd6625-abfa-4abe-b7fc-1f7f91f8ffdf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:22.908040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 465/917] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 08:53:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:22.914289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:22.946148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259e1e-bab0-40f2-b727-88480490f4c3 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.946465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259e1e-bab0-40f2-b727-88480490f4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.946688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259e1e-bab0-40f2-b727-88480490f4c3 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.948771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 466/918] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:22.953787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:22.954342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:22.954921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:22.975419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:22.976956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:22.995935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:23.032222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d773b6d-5e46-4c2b-885a-d905488d6e99 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:23.052652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 467/919] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:23.071278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:23.071881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:23.072163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:23.072427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:23.073094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:23.074453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:23.149647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 453/920] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:23.328188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:23.435419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427242-c6a7-4439-a207-4e594aa6dd6a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:23.477931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:23.478927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:23.479426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:23.479913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:23.480709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:23.482251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:23.492150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:23.493413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:23.494271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:23.496645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 468/921] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:23.504192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:23.522222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:23.620640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab39be-9309-4c83-9da3-fbee0ec338ce None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:23.657301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:23.657911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:23.658189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:23.658463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:23.659090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:23.659418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:23.660788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:23.665994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 454/922] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:23.671755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:23.689387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:23.780787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66f0d4-19a6-4d5e-8160-c276d8b96909 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:23.818712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:23.819695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:23.820203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:23.820666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:23.821571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:23.822064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:23.823635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:23.829143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 469/923] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:23.835175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:23.918768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:23.940519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd90a-e468-4e07-87a3-d1efea14287d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:23.976593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:23.977589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:23.978085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:23.978609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:23.979607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] RBAC: Adding query filter params (interface=public, service_id=8c02606c154e43c4ad0428ea7e208997, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:23.980115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:23.981732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f1c00-f210-478a-88a4-8ce3d3d1c5bd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:23.987265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 455/924] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:23 2022] GET /identity/v3/endpoints?service_id=8c02606c154e43c4ad0428ea7e208997&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:23.994036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.046495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660726f8-dd14-4fc9-b11b-5f3c7d8acbc3 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.046809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660726f8-dd14-4fc9-b11b-5f3c7d8acbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.047053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660726f8-dd14-4fc9-b11b-5f3c7d8acbc3 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.049772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 456/925] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:24.053688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfc93-d32b-4cfa-84c9-ee493cd8d1bb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.053973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfc93-d32b-4cfa-84c9-ee493cd8d1bb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.054252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfc93-d32b-4cfa-84c9-ee493cd8d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.074647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ddddfc93-d32b-4cfa-84c9-ee493cd8d1bb None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:24.075883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ddddfc93-d32b-4cfa-84c9-ee493cd8d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:24.081866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:24.109561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776ffa1-6a18-401e-bc4e-50014cca3a15 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.146682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:24.147328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.147604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.147928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.148522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:24.150089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a59111-c654-45a5-ac82-a4b77e4a1d95 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:24.156876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 470/926] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:23 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 08:53:24.175351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 457/927] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:24.222379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938fca-8214-47c8-8677-bf1a7de8f8ba None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.222676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938fca-8214-47c8-8677-bf1a7de8f8ba None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.222926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a938fca-8214-47c8-8677-bf1a7de8f8ba None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.224955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 471/928] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:24.230685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.230893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.231175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.254773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:24.257218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:24.335538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 458/929] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:24.448606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a938fca-8214-47c8-8677-bf1a7de8f8ba None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.568934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a938fca-8214-47c8-8677-bf1a7de8f8ba None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.605439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.607826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:24.608715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.608994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.609266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.609440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:24.623387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:24.629754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:24.640524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 472/930] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:24.641504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.645727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:24.719017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea620-1c30-42c7-b5ee-0a06c0fe23a6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.761195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:24.762155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.762493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.762778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.763420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:24.764823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:24.777584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:24.779053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 08:53:24.780953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:24.782505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18493-7796-43a3-9a84-62370df909a1 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.787149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 459/931] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:24.787149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.828627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:24.829678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.830668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.831207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.832092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:24.832913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:24.835572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:24.846463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 473/932] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:24.852999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:24.864179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:24.909814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8c8d9-59d0-4495-a7fd-d6d12ddf4632 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.921215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:24.960041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:24.961444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:24.962205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:24.962742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:24.963627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:24.963931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada1854-62c0-4ea7-a2ba-07665c903abf None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:24.964357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:24.966135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:24.978036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 460/933] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:24.978687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.006283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.006927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.007219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.007491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.019268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.021440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:25.038033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:25.039051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:25.039992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:25.039992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:25.039992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:25.043290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 474/934] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:25.045325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.120890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d4e8-5e29-4708-b85e-c426a1b7657d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.162833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.164388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d63c1-cb6f-447b-834e-8bd87a6d6130 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.173365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.174335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.175347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.175836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.177175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:25.177979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.179918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.183463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.187840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 461/935] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:25.192450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.213677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.214703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.215127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.215479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.216198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:25.216731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.218665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405430dc-2122-4196-8ab4-f53f63ca081d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.233367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 475/936] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:25.234688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644a38-a7f1-4883-a05e-a149192c98a7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.235047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644a38-a7f1-4883-a05e-a149192c98a7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.235423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644a38-a7f1-4883-a05e-a149192c98a7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.236826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 476/937] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:25.240521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644a38-a7f1-4883-a05e-a149192c98a7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.314533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e167d-60f1-467a-a281-ad880fa9f2e8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.349533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644a38-a7f1-4883-a05e-a149192c98a7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.351987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.352597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.352880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.353162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.353825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.355237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.386276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.386894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.387197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.387513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.394398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.396516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-2616918f-0405-449a-a76b-f77004e4e940 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:25.407846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:25.414165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 477/938] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.415252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.415995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.428055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:25.429246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:25.457721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.475138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 462/939] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:25.475138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.507007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 478/940] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:25.555389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.619699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b9a0b-48b5-493d-a5be-e38e8d860294 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.659048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.659686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.659969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.660243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.660963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:25.661310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.662820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.681126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 463/941] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:25.687464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.794947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cc283-758d-4477-b161-84c69dc951ad None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.804472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.832524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.834035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.834499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.834942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.846837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.848601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:25.864219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:25.865888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:25.866784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:25.866784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:25.866784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:25.869065 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 479/942] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:25.879028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:25.893735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.921880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4b729-39f2-4a24-b5ec-52fa856c461d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:25.932940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:25.960406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:25.961109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:25.961461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:25.962217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:25.962610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:25.964487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:25.967197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING keystone.server.flask.common [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 08:53:25.992486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30df81-28ff-4959-9ab1-64a01ed5cf99 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.024528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.sql.core [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 08:53:26.024528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 08:53:26.024528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [parameters: {'id': 'beecd298f1794a93baaf580f06120698', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 08:53:26.024528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81634) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 08:53:26.031376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.032088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.032369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.032720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.033370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:26.034740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.037150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id beecd298f1794a93baaf580f06120698.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id beecd298f1794a93baaf580f06120698. Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:53:26.037540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 08:53:26.038472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application yield resource Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise self.value Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application session.commit() Sep 01 08:53:26.039510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.flush() Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application compat.raise_( Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 08:53:26.041133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 08:53:26.042064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:53:26.042881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [parameters: {'id': 'beecd298f1794a93baaf580f06120698', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.043786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:26.044782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id beecd298f1794a93baaf580f06120698. Sep 01 08:53:26.047024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.050495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 464/943] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Sep 1 08:53:25 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:26.054376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 480/944] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:26.057646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.061951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.151436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:26.178576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39a232-c130-4901-bbec-06656f927ca8 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.186756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f744e6-e5e4-4824-9dd4-38452a8539db None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.217821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.218289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.218568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.218917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.226404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.228118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.228974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.229595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.229874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.230147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.236917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.238545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3a00037e-5643-4741-80b7-eac8c10da80b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:26.241228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:26.242111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:26.243319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:26.247920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 481/945] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:26.247920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052720-9248-414f-a795-3dc86d253925 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.247920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052720-9248-414f-a795-3dc86d253925 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.247920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052720-9248-414f-a795-3dc86d253925 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.247920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 482/946] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:26.250780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38052720-9248-414f-a795-3dc86d253925 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f9eb9e35-9ef3-4c03-a87d-62691c88bb96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:26.251288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:26.252166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.253089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 465/947] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:26.253589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.254075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.254075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.272512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:26.274023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:26.293892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:26.355487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 466/948] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:26.356304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.366694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38052720-9248-414f-a795-3dc86d253925 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.402451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.403403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.403880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.404340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.405251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:26.405746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.407376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.409384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:26.421714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 483/949] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:26.427510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.482389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155191f8-55e4-423e-960d-9ae38f34d38c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.530963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.531630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.531913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.532190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.533132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:26.533442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.535172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.556902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aaf91-fb23-4117-8a80-29a718cad3ef None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.571351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 467/950] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:26.600220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.600899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.601173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.601443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/e9d0bba06c9c4425a0e96641dab46c19/roles/f8fd612f0e494586a34c8c2079efa989` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.603778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=e9d0bba06c9c4425a0e96641dab46c19, role_id=f8fd612f0e494586a34c8c2079efa989)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.605238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.641342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.659468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:26.680388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 484/951] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:26 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/e9d0bba06c9c4425a0e96641dab46c19/roles/f8fd612f0e494586a34c8c2079efa989 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 08:53:26.770255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee2da-45c0-44a3-82e7-18b0b850c068 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.814265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.814942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.815259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.815536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.816130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.817635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:26.830402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:26.831651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:26.832562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 468/952] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:26.838138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:26.903195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:26.952535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0706bd-4e30-426a-b47e-6e82fdd95dfa None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:26.988039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:26.988645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:26.988919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:26.989187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:26.990460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:26.990763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:26.992151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:26.997757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 485/953] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:27.003259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:27.085926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:27.108337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c454b-6002-4248-aa58-7553c2fe9fab None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:27.144217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:27.145183 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:27.145699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:27.146172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:27.147026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:27.147573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:27.149153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a361f21-7bc0-479e-8159-9060edafdd99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:27.154927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 469/954] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:27.160239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:27.267922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8a515-11ee-4481-b267-0956790be7d2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:27.287728 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:27.304286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:27.304883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:27.305148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:27.305416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:27.306082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:27.307475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dc5ce-3714-4d3b-bc73-9b1d63b4f660 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:27.377141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 486/955] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:27.547362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b903-362e-43a9-8445-81c01bd5e74a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:27.547714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b903-362e-43a9-8445-81c01bd5e74a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:27.547921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225b903-362e-43a9-8445-81c01bd5e74a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:27.549549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 470/956] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:27.555808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:27.556163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:27.556523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:27.575819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:27.577118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:27.651145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 487/957] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:27.920143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225b903-362e-43a9-8445-81c01bd5e74a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:28.026645 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225b903-362e-43a9-8445-81c01bd5e74a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:28.061410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:28.062030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.062310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.062582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] PATH_INFO: `/v3/services/compute` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.063180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:28.064540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:28.074078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:28.075180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:28.076094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 471/958] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:28.083586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:28.185257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:28.189229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512941a3-3825-457a-9f0b-4a79be83324a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:28.235800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:28.236412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.236691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.236960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.237585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:28.237883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:28.239260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e7953-18d7-4204-b75a-269d38d8e88d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:28.245018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 488/959] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:28.250600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:28.262543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:28.358662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ceb172-f047-4938-860e-5eace83c46de None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:28.397412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:28.398032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.398310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.398581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.399241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:28.399539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:28.400916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d865d5-9c33-428e-9996-b65770bb4303 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:28.406126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 472/960] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:28.727040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499cc5-cb70-4da9-99d7-d428233f9f5e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.727458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499cc5-cb70-4da9-99d7-d428233f9f5e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.727636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0499cc5-cb70-4da9-99d7-d428233f9f5e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.729265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 489/961] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:28.734474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.734761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.735036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.758054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:28.759286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:28.812319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8871-acd1-4dd1-80df-919557214ad0 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.812495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8871-acd1-4dd1-80df-919557214ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.812772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa8871-acd1-4dd1-80df-919557214ad0 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.814247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 490/962] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:28.821117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:28.821653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:28.822560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:28.832542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:28.839890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 473/963] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:28.850805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:28.850805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:28.920057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 491/964] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:29.115786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.207200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.224999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de848655-6300-4c6b-98c1-ec050e1d8102 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.267331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.267962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.268311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.268657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.279534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:29.284457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:29.285431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:29.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:29.291139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 474/965] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:29.292119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.325848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.335195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a15fc-9d5c-4907-a7b3-0974a66fcd32 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.363121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.374982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.375460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.375807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.376169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.376913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.378478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.381476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING keystone.server.flask.common [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 08:53:29.411889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5eddb-2b91-44c0-ba61-87c52ec2f636 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.438561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.sql.core [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 08:53:29.438561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 08:53:29.438561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [parameters: {'user_id': '42ced06c9db74cd7a84a1ed75ec28b63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 08:53:29.438561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81633) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 08:53:29.447236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.447850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.448130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.448423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.449088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:29.449384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.450753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6edf1d-ed91-4b8c-958e-fd175c5496f9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-944c7001-9979-407c-9a7d-6c7f0a601035 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application result.read() Sep 01 08:53:29.455473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:53:29.456391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application yield resource Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application next(self.gen) Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise self.value Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application session.commit() Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 08:53:29.457399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self.session.flush() Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application compat.raise_( Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 08:53:29.459939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 08:53:29.460930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application conn.query(q) Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application result.read() Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 08:53:29.463499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [parameters: {'user_id': '42ced06c9db74cd7a84a1ed75ec28b63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:29.465786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 08:53:29.468158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 492/966] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 08:53:29 2022] POST /identity/v3/users => generated 160 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbabb9-b839-4027-96b0-4903fbc23cc3 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbabb9-b839-4027-96b0-4903fbc23cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.471670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbabb9-b839-4027-96b0-4903fbc23cc3 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.472985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 493/967] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:29.472985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbabb9-b839-4027-96b0-4903fbc23cc3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.473150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 475/968] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:29.473150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.473150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.473150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.493674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:29.494905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:29.571143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 476/969] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:29.575976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.586786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbabb9-b839-4027-96b0-4903fbc23cc3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.622489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.623132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.623430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.623696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.633652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.635228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:29.647404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:29.648404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:29.649383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 494/970] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:29.655490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.657641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.684255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.716132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6f4bc-8242-4299-b9cc-23241a2bf326 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.753078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.753859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.754146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.754523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.765628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.767614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.777616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83af5f-1b4d-4c3f-8324-0398cb2c3404 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:29.783771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:29.784788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:29.785727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:29.785727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:29.785727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:29.790310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 477/971] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:29.791319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.819337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.820283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.820806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.821409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.822416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:29.822943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.824905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.862529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 495/972] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:29.863738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:29.913839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb25b4d-955d-4568-a52b-befae9314108 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:29.947523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.953096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:29.953810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:29.954163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:29.954515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:29.955381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:29.955761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:29.957241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:29.969184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:29.976300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 478/973] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:29.979145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.007606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a142-5463-46c1-a95e-6afeb0b112ab None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.054405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.055047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.055360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.055672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.056287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.057579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:30.068200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:30.069335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:30.070210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 496/974] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:30.074961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.119212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b521cdf-a462-41a8-8cf5-dfba26c87d8d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.162784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.163466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.163749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.164076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.174955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.177886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:30.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:30.192064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:30.193617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 479/975] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:53:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:30.198512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.205626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f1d99-700b-4f4d-a434-77d31ae0fb0b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.226620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.247709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.248375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.248652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.248917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.249531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:30.249823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.251247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.260913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 497/976] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:30.261593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.264569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.310439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158d5b6-1c01-4ab2-af52-5e5e3205d6f6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.347866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.348474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.348785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.349588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.350311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:30.350610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.352065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039cd2d-6b02-4401-89d1-435b571805a6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.368543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9702dda-65b7-4874-b9a8-2079888e46ab None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.369394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 480/977] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:30.370573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a024ee9-94ed-4378-8fdd-35267433c515 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.370854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a024ee9-94ed-4378-8fdd-35267433c515 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.371147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a024ee9-94ed-4378-8fdd-35267433c515 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.372338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 481/978] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:30.376385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a024ee9-94ed-4378-8fdd-35267433c515 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.403328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.403933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.404212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.404482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.405106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:30.405405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.406787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46ec3e-8951-4651-8090-54dd66ee56d4 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.412998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 498/979] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:30.414196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.414421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.414691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.433022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:30.434529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:30.459383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.479561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a024ee9-94ed-4378-8fdd-35267433c515 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.495502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.503566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 499/980] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:30.504435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.517240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.517848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.518126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.518401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.589047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.590725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.624511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4b6e9-816b-4aef-ab2f-5edfe6612086 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.662876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.663784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.664062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.664396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.665033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] RBAC: Adding query filter params (interface=admin, service_id=8c02606c154e43c4ad0428ea7e208997, region_id=RegionOne) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:30.665332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.666688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.675609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.676367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 500/981] 10.222.0.46 () {56 vars in 1249 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/endpoints?service_id=8c02606c154e43c4ad0428ea7e208997&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:30.677159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.695830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 482/982] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:30.701235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.744303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.788344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795cc45-a969-4f6d-b312-57d78f5fef87 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.806240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27de98-3ae9-4d03-a757-6ecb2bcf64c0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.823045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.823670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.823949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.824221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.824818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.826185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.831962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 501/983] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 08:53:30.832712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.841685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:30.842334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:30.842613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:30.842886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:30.858702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:30.860252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:30.878286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:30.879425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:30.881682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:30.881682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:30.881682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:30.881682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 483/984] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:30.885901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:30.930083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:30.954590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0b691-1955-4941-a60b-746199b0aaa5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:30.986965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:31.001837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:31.002547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:31.002881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:31.003173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:31.003824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:31.005277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:31.006858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b38320-5aaf-4f2d-92d5-ec2cc7b42e0f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:31.042829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:31.043464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:31.043749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:31.044023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:31.044664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:31.044975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:31.046337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:31.058187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 484/985] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:31.063679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:31.131159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 502/986] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:30 2022] POST /identity/v3/services => generated 242 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:31.165186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:31.174094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be0ed-8410-4a6e-b222-98c2f88c5852 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:31.221688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:31.222240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:31.222515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:31.222782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:31.230762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:31.232274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:31.245732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:31.246713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:31.248022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:31.248818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 485/987] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:31.253075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:31.365792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb348a0a-88ad-4818-a6a9-8c971a852eaf None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:31.400678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:31.401284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:31.401562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:31.401828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:31.402505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:31.402804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:31.404257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee61050-c8bf-45df-b34b-52517890efb1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:31.417358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 503/988] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:31.423122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:31.525836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521e205-14d9-4256-b532-11e16b317fb6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:31.570139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:31.570746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:31.571021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:31.571331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:31.572117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] RBAC: Adding query filter params (role.id=f8fd612f0e494586a34c8c2079efa989, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=e9d0bba06c9c4425a0e96641dab46c19) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:31.572429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:31.573858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17820c2-81f8-4c85-a6a7-e2dcd8dbd6a0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:31.579952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 486/989] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:31 2022] GET /identity/v3/role_assignments?user.id=e9d0bba06c9c4425a0e96641dab46c19&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=f8fd612f0e494586a34c8c2079efa989 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:32.766856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3350a-272f-4c65-bf14-ee438aa6496e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:32.767437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3350a-272f-4c65-bf14-ee438aa6496e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:32.767437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3350a-272f-4c65-bf14-ee438aa6496e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:32.768921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 504/990] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:32.773993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776930-55c6-4a10-9858-a938e09bb20a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:32.774432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776930-55c6-4a10-9858-a938e09bb20a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:32.774526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a776930-55c6-4a10-9858-a938e09bb20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:32.792521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9a776930-55c6-4a10-9858-a938e09bb20a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:32.793693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9a776930-55c6-4a10-9858-a938e09bb20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:32.803635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:32.863272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 487/991] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:33.107185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c317-0bfd-4f22-8322-84669603f063 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.107525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c317-0bfd-4f22-8322-84669603f063 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.107722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c317-0bfd-4f22-8322-84669603f063 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.109154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 505/992] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:33.114011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.114311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.115215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.130979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c317-0bfd-4f22-8322-84669603f063 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.133505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:33.135934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:33.182304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.205755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 488/993] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:33.240822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c317-0bfd-4f22-8322-84669603f063 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.276367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.277219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.277640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.278072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.278787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.280319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:33.289760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:33.290983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 08:53:33.292085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:33.294162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 506/994] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:33.300026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.375343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.400634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91860cc-1825-4e0f-90f3-e87ee9dfba0a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.435194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.435794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.436068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.436333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.436950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:33.437246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.438603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.447172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 489/995] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:33.449387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.475612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.558611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b91a0-df9c-44c2-84b2-1ef1ec786134 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.570666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.577602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb519-0453-4673-916d-d1801374b9e3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.596590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.597223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.597734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.597998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.598619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:33.598947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.600321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2b20a-5914-472c-8746-e4b32e6fae63 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.604971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 507/996] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:33.606085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb641-d0c6-4ad5-a439-48db3c102d8d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.606361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb641-d0c6-4ad5-a439-48db3c102d8d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.606622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb641-d0c6-4ad5-a439-48db3c102d8d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.607833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 508/997] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:33.611979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb641-d0c6-4ad5-a439-48db3c102d8d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.612763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.613390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.613672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.613947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.614569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.616247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d57e0527-52ed-42d9-9cef-2dfd34397fa1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:33.625438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:33.626482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:33.627388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 490/998] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:33.628206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.628206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.628206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.636411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.648378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:33.649579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:33.713451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb641-d0c6-4ad5-a439-48db3c102d8d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.717448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 491/999] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:33.718288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.753865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.754830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.755395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.755970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.756838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.758655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.815396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d251bb-088b-43c6-97e2-8c6955fce93c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.831479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 509/1000] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:33.848065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.860497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:33.861118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:33.861396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:33.861668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:33.862310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:33.862612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:33.864067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:33.870636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 492/1001] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:33.881404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:33.910806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:33.986208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae40d8-89b1-46ad-a957-1345a8a036e9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:33.987526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:34.026326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.026936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.027247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.027523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.028700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:34.028700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.030523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.035357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 510/1002] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:34.041826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:34.111128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d868-0753-4b9d-98c9-197670484a6c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:34.146365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.147192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.147479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.147755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] PATH_INFO: `/v3/services/image` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.150133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.150133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:34.159360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:34.160428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:34.162361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 493/1003] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 08:53:33 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:34.163659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:34.168423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:34.169132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a8fe-0f4e-4d85-b4b0-040e9077c45f None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:34.222433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.224476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.224834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.225476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.226255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] RBAC: Adding query filter params (interface=public, service_id=75d05deacd0f47bb904c677b083e2cfc, region_id=RegionOne) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:34.226628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.228107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.233694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 511/1004] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:34 2022] GET /identity/v3/endpoints?service_id=75d05deacd0f47bb904c677b083e2cfc&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:34.239204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:34.244370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:34.301089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ced2-071f-432a-ae0a-24a0dba6f75b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:34.342159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.343741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.344021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.344294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.344927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] RBAC: Adding query filter params (name=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:34.345226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.346607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.359462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 494/1005] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:34.363779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:34.373734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e1a2-c96f-4165-83d6-2da09a8a29b9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:34.422897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.423452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.423735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.424017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.424723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.426189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a8a9b-e7c4-45e4-8ffd-54ebe4b3f641 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.438537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 512/1006] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:34 2022] GET /identity/v3/services => generated 1727 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 08:53:34.500879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36cba5-34f8-4f69-b60e-9a7545299bcc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:34.522871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:34.538973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:34.539623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:34.539900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:34.540169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:34.540798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] RBAC: Adding query filter params (type=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:34.541096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:34.542579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281660ea-4638-4201-b6f0-49b8504804da None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:34.547908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 495/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:35.851273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16dc9-b0f9-41bc-bf06-7cbfaa40405d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:35.851768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16dc9-b0f9-41bc-bf06-7cbfaa40405d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:35.851768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc16dc9-b0f9-41bc-bf06-7cbfaa40405d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:35.853218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 513/1008] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:35.858360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae8b92-9332-440c-b095-7e4834a7cb8d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:35.858755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae8b92-9332-440c-b095-7e4834a7cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:35.858934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae8b92-9332-440c-b095-7e4834a7cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:35.877332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-37ae8b92-9332-440c-b095-7e4834a7cb8d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:35.878516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-37ae8b92-9332-440c-b095-7e4834a7cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:35.946644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 496/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:36.214396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc16dc9-b0f9-41bc-bf06-7cbfaa40405d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:36.239270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:36.318243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc16dc9-b0f9-41bc-bf06-7cbfaa40405d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:36.352531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:36.353153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.353424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.353688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.354778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:36.356226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:36.357044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70fb5-c431-4dc1-8a73-77ccb10f437d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.357336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70fb5-c431-4dc1-8a73-77ccb10f437d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.357616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f70fb5-c431-4dc1-8a73-77ccb10f437d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.359174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 514/1010] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:36.359310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 497/1011] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:36.363257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:36.363911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac695-8bad-45e3-92cb-6d9bc200bee8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.364195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac695-8bad-45e3-92cb-6d9bc200bee8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.365006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac695-8bad-45e3-92cb-6d9bc200bee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.384109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e75ac695-8bad-45e3-92cb-6d9bc200bee8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:36.385306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e75ac695-8bad-45e3-92cb-6d9bc200bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:36.455090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 498/1012] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:36.456355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc641b9-b0c0-42ca-8bea-f3f5cdf3485d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.456642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc641b9-b0c0-42ca-8bea-f3f5cdf3485d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.456916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc641b9-b0c0-42ca-8bea-f3f5cdf3485d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.462648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 499/1013] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:36.463873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.464151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.464566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.467667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:36.478247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ddf20-74d4-4f8a-b168-61a3baa55986 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:36.483180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:36.484377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:36.513860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:36.514485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.514766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.515040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.515734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:36.517151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:36.529296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:36.558209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 500/1014] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:36.623144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 515/1015] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:36 2022] POST /identity/v3/projects => generated 311 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:36.725097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:36.827512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:36.828442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347662d4-c92e-4524-a0f3-85d025fefb5e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:36.866970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:36.867794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.868335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.868610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.869332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:36.870798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:36.881093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:36.882311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:36.883366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 501/1016] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:36.888397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:36.930559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:36.941510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2275d7-5b93-4644-9b13-8b88079179c2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:36.977153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:36.977672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:36.978140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:36.978419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:36.980068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:36.981443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:36.992350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83dda-6b9f-42fd-8342-29fbbd2dc2d4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:37.027285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:37.027890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:37.028170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:37.028444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:37.029116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:37.029421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:37.030781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:37.035499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 502/1017] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:37.041457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:37.072627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 516/1018] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:53:36 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:37.155216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510a965-156f-45a9-a636-260e9926aa8b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:37.157455 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:37.200430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:37.201441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:37.201926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:37.202417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:37.203325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:37.203830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:37.205421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81975b3-699c-48ee-8f3c-b5511075b18b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:37.211571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 503/1019] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 08:53:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:37.217177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:37.221983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:37.322291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b7af-400b-41bf-8bf2-2a59f3120042 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:37.356359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:37.356954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:37.357227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:37.357488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:37.358143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:37.359574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebffd4-fcce-4251-9d72-1b98122e174e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:37.432491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 517/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:38.603820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075a991-96e5-4792-be5e-3526ef320812 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:38.604359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075a991-96e5-4792-be5e-3526ef320812 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:38.604359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075a991-96e5-4792-be5e-3526ef320812 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:38.605714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 504/1021] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:38.611192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900ab9-2c0d-4f62-857e-1f35d11ea05a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:38.611600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900ab9-2c0d-4f62-857e-1f35d11ea05a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:38.611806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900ab9-2c0d-4f62-857e-1f35d11ea05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:38.631472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c4900ab9-2c0d-4f62-857e-1f35d11ea05a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:38.632714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c4900ab9-2c0d-4f62-857e-1f35d11ea05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:38.690356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:38.707843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 518/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:38.977534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075a991-96e5-4792-be5e-3526ef320812 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.031230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7bf-0169-4c73-ad63-cefcb4913d84 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.031664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7bf-0169-4c73-ad63-cefcb4913d84 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.031945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7bf-0169-4c73-ad63-cefcb4913d84 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.033295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 519/1023] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:39.039652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.040219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.040219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.059725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:39.060919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:39.094411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075a991-96e5-4792-be5e-3526ef320812 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.131718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.132337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.132610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.132894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.133983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:39.135154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 520/1024] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:39.135438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:39.139140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 505/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:39.142429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.246130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa37965-0870-423a-b6db-fb5ff565cee9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.262812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:39.280465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.281130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.281411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.282282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.282388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:39.283875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02252c-95ac-434c-ab9f-1072a48e3813 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:39.399537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 521/1026] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:53:39 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:39.414097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.430145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0e69-0d30-4232-966c-da31a1eb64e8 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.430500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0e69-0d30-4232-966c-da31a1eb64e8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.430686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0e69-0d30-4232-966c-da31a1eb64e8 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.432253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 522/1027] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:39.437673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.437955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.438220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.452783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:39.459016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:39.460547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:39.533521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 523/1028] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:39.534813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d0ea0-7a9b-4041-972c-ee068b48fa34 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.579953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.580826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.581109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.581381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] PATH_INFO: `/v3/services/image` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.581955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:39.584017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:39.592914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:39.593927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:39.594810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:39.595703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 506/1029] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:39.600914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.704798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a19e2e-d896-4564-a782-0485f4552f2f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.706329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:39.742312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.742914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.743480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.743746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.744360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] RBAC: Adding query filter params (name=image) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:39.744656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:39.746002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:39.750884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 524/1030] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:39.757060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.805615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.865326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:39.872397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13469-4254-489c-87f3-1bdf293dfebd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.906670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.907307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.907591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.907869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.908495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] RBAC: Adding query filter params (type=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:39.908801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:39.910179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:39.915814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc72f36-d1b4-4530-aa64-e77054c4d96c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:39.919249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 507/1031] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:39.921683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:39.956967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:39.957593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:39.957869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:39.958137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:39.969318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:39.974970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:39.975960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:39.976829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:39.980960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 525/1032] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:39.981754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.009167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:40.037627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638e24-ebbd-4baf-8dd0-73b91cb2c669 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.077880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.078559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.078849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.079461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.080446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] RBAC: Adding query filter params (interface=public, service_id=0b6f3ab663884284b028283fb900156d, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:40.080763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.082627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.085892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e0f7d-0535-454a-bb16-569cbd45a2ec None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.094480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 508/1033] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/endpoints?service_id=0b6f3ab663884284b028283fb900156d&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:40.096142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.123703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.124304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.124588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.124962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.125473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:40.125627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:40.125930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.127327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.136692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 526/1034] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:40.142370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.216426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3302fa-c5e7-469f-ab37-9e0a218fd0d1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.241893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:40.251751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.252358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.257615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.257615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.257615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.257911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c5fa-99d5-4f0a-83c5-5d31d12eb9d9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.258003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.269190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 509/1035] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 08:53:40.302686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.303354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.303626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.303890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.316267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.318352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:40.334929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:40.336025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:40.336950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:40.336950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:40.336950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:40.341308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 527/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:40.342340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.437638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:40.452775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fefb9-20f2-4742-b841-e2d517db8dba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.487546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.488152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.488433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.488704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.489337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:40.489694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.491642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.506248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 510/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:40.512289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.620001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f85b8-11b1-44ec-9bc5-50de6078a3f0 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.653031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.653643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.653917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.654183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.662162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.663805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.666316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:40.675616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:40.679770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:40.680766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 528/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:40.683237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.783932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda7a82-d750-4998-a8b8-36f6dac152fe None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.818282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.818980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.819286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.819556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:40.820277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:40.820587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:40.822000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:40.844285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 511/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:40.846051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:40.948417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712de0-eea5-4010-867c-26b2371f2a3b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:40.980954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:40.981550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:40.981824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:40.982127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.046968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.049817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.051294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.153549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 529/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:41.154866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.159703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.260080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135293e0-9c7e-4feb-a097-493232ac0d29 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.295283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.295913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.296247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.296531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.307463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.308914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ca2bfcf6-7ee4-4be8-92db-8ac78ad6130f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:41.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:41.324835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:41.325838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:41.325838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:41.325838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:41.325838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 512/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:41.331815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.413717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6774ec3-fb49-4784-b788-896f9cd7627f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.414043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6774ec3-fb49-4784-b788-896f9cd7627f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.414268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6774ec3-fb49-4784-b788-896f9cd7627f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.415772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 513/1042] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:41.420343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.420629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.420909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.434927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.437993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5559a5d-b0b7-40f4-a39e-a66fa1e1f4f7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.439965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:41.441154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:41.476108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.476712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.476992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.477261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.478015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:41.478432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.479822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.496506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 530/1043] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:41.497844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.498314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.519214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 514/1044] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:41.606441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7871d-69df-481c-b515-7a0f555c2b46 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.640123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.640736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.641007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.641272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.648500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.649939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:41.661251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:41.662807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:41.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:41.667539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 531/1045] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:41.668745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.770457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94948710-a1c4-40a1-aa75-f6567d48c20b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.783736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.808528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.809185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.809413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.810008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.810368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:41.810675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.810913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.812158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.829680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 515/1046] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:41.830956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.883989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:41.903740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7247a-b4f1-49ed-b3b0-a17692294f29 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.939013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.939422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec16c-4e83-4f45-bcd2-549af9e10b46 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:41.939671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.939945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.940217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.951689 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:41.956875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:41.957840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:41.958840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 532/1047] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:41.964047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:41.977809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:41.978412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:41.978690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:41.978970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:41.979718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] RBAC: Adding query filter params (role.id=f8fd612f0e494586a34c8c2079efa989, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:41.980026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:41.981552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:41.987083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 516/1048] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=f8fd612f0e494586a34c8c2079efa989 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:42.060990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.081900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eefaaa-0503-460f-aee4-8eab3a2267b5 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.120894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.121511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.121784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.122053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.122718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:42.123020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.124411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c564d-417e-48be-890b-63f5cf560fe2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.138471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 533/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:42.139694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.193206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9431-bf65-4c25-9242-328f37cfcab0 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.193545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9431-bf65-4c25-9242-328f37cfcab0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.193874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9431-bf65-4c25-9242-328f37cfcab0 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.195515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 534/1050] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:42.200115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.200393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.200735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.220300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:42.221487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:42.258854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130bb9-c83f-4236-ac99-a79566c3145d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.269890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.292223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 535/1051] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:53:42.295223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.295369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.295620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.295859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] PATH_INFO: `/v3/users/4f92c889d74d487298b192cd719e48b6` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.310677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] RBAC: Authorizing `identity:get_user(user_id=4f92c889d74d487298b192cd719e48b6)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.312195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.313990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 517/1052] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/users/4f92c889d74d487298b192cd719e48b6 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:42.320492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.389284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.418484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d7d89-a9cf-49b3-a8c4-94db05aabcc6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.453838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.454441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.454715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.454982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.520425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.522019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.562087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.596580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.634318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 536/1053] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:42.635557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.672409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a29-c935-41e0-8a58-77b41b4ecb1f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.713933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.714828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.715137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.715415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] PATH_INFO: `/v3/services/image` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.716157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.718195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:42.729963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:42.731090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:42.736119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 518/1054] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:42.736971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb57fdf-775b-47b7-b62f-08486d684d85 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.737902 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.770084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.771508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.772081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.772354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.772622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.773879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.776103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.782650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 537/1055] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:42.783572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.828532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:42.844502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b57110-d2cd-4ef1-a9ce-7f4504ef747b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.881895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.882335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.882616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.882888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.883538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:42.883835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.885214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.890289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 519/1056] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:42.892642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcdc8-eadb-4581-bce0-0651274c7bcf None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:42.896392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:42.931112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:42.931826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:42.932112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:42.932382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:42.933099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d, user.id=4f92c889d74d487298b192cd719e48b6) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:42.933402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:42.934812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:42.951147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 538/1057] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/role_assignments?user.id=4f92c889d74d487298b192cd719e48b6&scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:43.014815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:43.022219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9af6-02fe-4f53-81d1-135a1312aec3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:43.059446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:43.060542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:43.060927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:43.061308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:43.062078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] RBAC: Adding query filter params (type=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:43.062479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:43.064202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114de06-b9b9-4b24-a9dc-45baac356557 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:43.074858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 520/1058] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:43.076591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:43.182120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10073066-5f65-4054-a386-ad2ba4219f9e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:43.184390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:43.216197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:43.216872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:43.217218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:43.217557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:43.218323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:43.220230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3033-4b4a-4dd9-afba-d8cbba58245e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:43.318111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 539/1059] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:53:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:53:43.939040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981289-4e35-47c6-89cc-9e0f9a68e963 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:43.939560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981289-4e35-47c6-89cc-9e0f9a68e963 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:43.939560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981289-4e35-47c6-89cc-9e0f9a68e963 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:43.940923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 521/1060] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:43.947005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:43.947451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:43.947625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:43.966049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:43.967290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:44.045740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 540/1061] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:44.310695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d981289-4e35-47c6-89cc-9e0f9a68e963 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:44.417987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d981289-4e35-47c6-89cc-9e0f9a68e963 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:44.453505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:44.454133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.454414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.454688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.465938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:44.470849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:44.471940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:44.476226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 522/1062] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:44.478888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:44.568611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:44.582880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad742-261a-4aec-ab31-4386748fdd69 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:44.616722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:44.617324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.617600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.617870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.618783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:44.619574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:44.621691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:44.629844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 541/1063] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:44.635535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:44.658563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:44.738538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1f359-5c48-4b68-95c4-d05c67706992 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:44.772375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:44.772997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.773277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.773548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.784037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:44.785477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-55ff0b07-a444-4870-a313-5684d17cddfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:44.800458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:44.801468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:44.802377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:44.802377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:44.802377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:44.803424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 523/1064] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:44.807431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:44.830985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3e3d-506d-41ef-9259-98e4756105cd None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.831202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3e3d-506d-41ef-9259-98e4756105cd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.831473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3e3d-506d-41ef-9259-98e4756105cd None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.832861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 524/1065] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:53:44.839314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255667-000e-4165-943e-0798754ffb0c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.839597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255667-000e-4165-943e-0798754ffb0c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.839866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255667-000e-4165-943e-0798754ffb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.857980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-52255667-000e-4165-943e-0798754ffb0c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:44.859187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-52255667-000e-4165-943e-0798754ffb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:44.921201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcf91f-c634-4521-b7ef-e95bb7182661 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:44.932460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 525/1066] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:44.959630 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:44.960305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:44.960489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:44.960746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:44.961375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:44.961668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:44.963015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:44.979496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 542/1067] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:44.981168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52255667-000e-4165-943e-0798754ffb0c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:44.983796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.084735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52255667-000e-4165-943e-0798754ffb0c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.119711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.120339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.120619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.120891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.128132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.129582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:45.141478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:45.142458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:45.143514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:45.147265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 526/1068] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:53:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:45.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.160123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.199644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.261661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b7de6-ae19-4486-8f5b-8f971a34e668 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.262416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.296506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.297105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.297384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.297653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.298376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:45.298675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.300120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4cd3-13d0-4054-8785-c4018128e274 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.302634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0de6-d8af-4de6-b079-b28d4ff1c54a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.318478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 543/1069] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:45.319833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980409-6d35-4261-916b-b1f2f70b0455 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.320117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980409-6d35-4261-916b-b1f2f70b0455 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.320388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d980409-6d35-4261-916b-b1f2f70b0455 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.321752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 544/1070] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:45.323935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d980409-6d35-4261-916b-b1f2f70b0455 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.337677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.338230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.338505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.338778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.348732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c35d40c8-1f4a-4eb0-9119-63d7a3a8098a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:45.354335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:45.355390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 527/1071] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:45.361411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.362188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.362188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.375007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:45.376220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:45.391511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.425922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d980409-6d35-4261-916b-b1f2f70b0455 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.442394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 528/1072] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:45.443901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.462755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.463417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.463696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.463965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.474049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.475831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:45.491840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:45.492860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:45.494045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:45.494045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:45.494045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:45.494045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 545/1073] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:45.498745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.501544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.557755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf50b3-b87e-4830-83ed-c8e185e91050 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.596013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.596644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.596923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.597197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.597892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:45.598209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.600238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.608588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 529/1074] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:45.609647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a58e-8d82-4536-9a01-7917541bfa3e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.614029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.628546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.646375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.647132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.647223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.647493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.648130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:45.648423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.649901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.663217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 546/1075] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:45.668511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.730053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4c84-85fe-479d-9774-660a60f2a85d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.745928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.768081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.768701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.768979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.769246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] PATH_INFO: `/v3/users/4f92c889d74d487298b192cd719e48b6` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.769804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562b496-0788-42ed-b4fa-5a3ac53e32db None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.770358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] RBAC: Authorizing `identity:get_user(user_id=4f92c889d74d487298b192cd719e48b6)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.772021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.773944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 530/1076] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/users/4f92c889d74d487298b192cd719e48b6 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:45.775032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.804962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.805584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.805860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.806129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.812710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.814137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:45.825286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:45.826361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:45.827313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 547/1077] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:45.827816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.870191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.876431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af435bc-393a-42ab-a9c6-9f0a1b53f8bb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.909200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.909786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.910064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.910334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.930937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2624fc-2686-4041-8868-74258fd5b05d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:45.959516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:45.964434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:45.965042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:45.965316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:45.970453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:45.970453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.970453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:45.970453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 548/1078] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:45.970453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:45.982137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:45.983630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.071487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fa373-c812-4be3-bf49-830e44dbadc7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:46.078860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:46.080926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 531/1079] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:46.081849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:46.106869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:46.107571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:46.107915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:46.108256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:46.108937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:46.109239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:46.110660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.127868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 549/1080] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:46.127868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:46.163613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:46.180687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563ec53-1032-4399-b9b9-5566bd983f8b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:46.214294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:46.214938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:46.215260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:46.215536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d/users/4f92c889d74d487298b192cd719e48b6/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:46.217414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] RBAC: Authorizing `identity:create_grant(project_id=3061a4ceb35b4e8b85954fc4adf5b83d, user_id=4f92c889d74d487298b192cd719e48b6, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:46.218844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.225901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115981c-3c64-4c57-b47c-9da51c79cbd2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:46.258737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:46.259381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:46.259659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:46.259930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:46.270141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:46.272164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:46.286052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:46.288010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:46.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:46.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:46.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:53:46.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 550/1081] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:46.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:46.306806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:46.310773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 532/1082] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:46 2022] PUT /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d/users/4f92c889d74d487298b192cd719e48b6/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:53:46.311640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:46.406340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42f873-7eee-41d5-9e20-964378470ac7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:46.430919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8932b6d-1d4b-4780-affd-4e7be1c2775c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:46.442523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:46.443792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:46.444075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:46.444352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/f8fd612f0e494586a34c8c2079efa989` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:46.446569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=d132894fe36047418e5cd8bebe04c387, role_id=f8fd612f0e494586a34c8c2079efa989)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:46.448117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1001f41-5572-4218-9083-cc6131a52d9f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.475516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:46.476129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:46.476409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:46.476683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:46.477324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:46.477625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:46.478997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5d168-4993-4c7e-84f8-36aca1675437 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:46.490681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 533/1083] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:46.512378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 551/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:46 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/d132894fe36047418e5cd8bebe04c387/roles/f8fd612f0e494586a34c8c2079efa989 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:53:48.217805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525dcf5-e8f7-4e69-bc09-6dac51635240 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.218337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525dcf5-e8f7-4e69-bc09-6dac51635240 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.218337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525dcf5-e8f7-4e69-bc09-6dac51635240 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.219871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 534/1085] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:48.225368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0aa09-8f34-4225-ab93-08d897a5327a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.225782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0aa09-8f34-4225-ab93-08d897a5327a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.226026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0aa09-8f34-4225-ab93-08d897a5327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.244033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c3f0aa09-8f34-4225-ab93-08d897a5327a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:48.245185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c3f0aa09-8f34-4225-ab93-08d897a5327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:48.309884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 552/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:48.410390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc021ed-7103-4fa3-9ec2-44670f7d0e42 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.410665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc021ed-7103-4fa3-9ec2-44670f7d0e42 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.410933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc021ed-7103-4fa3-9ec2-44670f7d0e42 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.413181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 535/1087] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:48.418188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.418479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.418757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.436884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:48.438059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:48.440034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:48.504280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 553/1088] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:48.576049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc021ed-7103-4fa3-9ec2-44670f7d0e42 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:48.578930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:48.676665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc021ed-7103-4fa3-9ec2-44670f7d0e42 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:48.709589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:48.710203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.710479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.710747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.721033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:48.726012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:48.727056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:48.728097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 536/1089] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:53:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:48.733019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:48.768815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:48.831709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:48.835517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db9b53-1108-4f01-8c8f-13862eb725e0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:48.869132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:48.869726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.870001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.870270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.870918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:48.871239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:48.871577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba295-7140-441f-a2d9-6ca54847b0f2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:48.872593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:48.877467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:48.881226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 554/1090] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:53:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:48.890532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:48.909662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:48.910285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:48.910563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:48.910835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:48.921029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:53:48.926298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:48.927438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 08:53:48.928340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:48.931394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 537/1091] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 08:53:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:48.933625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:48.995734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30d04-2495-45a6-b6cb-b8bf0b5b1d36 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.033821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.033821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.033821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.033821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] PATH_INFO: `/v3/users/4f92c889d74d487298b192cd719e48b6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.034296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] RBAC: Authorizing `identity:get_user(user_id=4f92c889d74d487298b192cd719e48b6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.035361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.037286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 555/1092] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:48 2022] GET /identity/v3/users/4f92c889d74d487298b192cd719e48b6 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.039913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13952-cd29-410f-918b-a722e3f3373c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.043002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.063750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.075568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.076165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.076441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.076709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.077359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:49.077655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.079012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.087570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 538/1093] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:53:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.093125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.109573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.143607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b61cfd-b680-4e2b-8418-f061ce332566 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.175722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.176317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.176592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.176865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.192000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6703-e239-4da8-be40-f10e2faab3ba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.226018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.226661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.226912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.227227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.237152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.238606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.246746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.248229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:49.255942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:49.257258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:49.258494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:49.258494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:49.258494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:49.259249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 539/1094] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:49.264023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.297065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.314943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.350030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 556/1095] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 08:53:49.356046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.362920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aa63b-bf2b-4d78-b3a9-696e52315f99 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.399707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.400306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.400579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.400844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.401470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:49.401768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.403145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.415347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 540/1096] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.420918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.459792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07458e7e-4dd4-461c-9f4f-8ca831dd9fc2 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.494578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.495234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.495514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.495780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.496882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.498321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.499976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 557/1097] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.506458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.514911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.520544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9747e2-40c4-43cd-8cbb-4ac209f39379 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.540768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.559294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.560018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.560196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.560457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.567822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.569281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:53:49.580870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:49.581930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:49.582955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 541/1098] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 08:53:49.587682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.608997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8f43b-6a38-4bd5-8817-f8195dd286a6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.643583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.644178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.644452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.644725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.645431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d, user.id=4f92c889d74d487298b192cd719e48b6) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:49.645734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.647540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.653170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 558/1099] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/role_assignments?user.id=4f92c889d74d487298b192cd719e48b6&scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.689429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb9dd2-acc9-4bf7-8354-201f8ebed5f8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.726596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.727250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.727535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.727808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.728560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:49.728936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.730411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:49.755003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 542/1100] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:53:49.756538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:49.856721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:49.859511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb38393-61dd-4b83-9007-acdaab13784b None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:49.894199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:49.894799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:49.895077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:49.895376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:49.962438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:49.963919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.057252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:50.060870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 559/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:50.066703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:50.083407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:50.168284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac9af3-b1ca-4637-ba20-27503032f978 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:50.201349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:50.201972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:50.202254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:50.202528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:50.212885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:50.214331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:53:50.229718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:50.231507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:53:50.232709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:53:50.232709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:53:50.232709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:50.232709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 543/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:50.236391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:50.334946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e58a0c-9ce1-4b21-944b-c58d6a2a3179 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:50.367852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:50.368702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:50.368980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:50.369248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:50.369872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:50.370171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:50.371591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.383143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 560/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:50.388957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:50.453983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:50.488536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7fe1-0f9c-424f-acbc-9c9c2b671736 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:50.522653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:50.523341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:50.523620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:50.523892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:50.530313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:50.531774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:53:50.542734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:53:50.543830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:53:50.544964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:53:50.547378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 544/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:53:50.549705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:50.608459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:50.649935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ade80-4287-409e-ad10-e0186a75538d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:50.682978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:50.683612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:50.683891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:50.684162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:50.684848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:50.685151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:50.686574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb937-174d-46ef-828a-7dce8b822c86 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.699123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 561/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:53:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:50.705378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:50.804971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b44dc-8ee6-4b24-9968-9060f1a3c9ae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:50.829304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:50.837818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] RBAC: Adding query filter params (role.id=f8fd612f0e494586a34c8c2079efa989, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=d132894fe36047418e5cd8bebe04c387) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:50.845401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54591a-d850-46ce-bdd6-c1a221790583 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:50.846857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 545/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:50 2022] GET /identity/v3/role_assignments?user.id=d132894fe36047418e5cd8bebe04c387&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=f8fd612f0e494586a34c8c2079efa989 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:51.596768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86460-0d60-4a1f-b713-3a0f968e733c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:51.597281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86460-0d60-4a1f-b713-3a0f968e733c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:51.597281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86460-0d60-4a1f-b713-3a0f968e733c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:51.598637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 562/1107] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:51.604120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005281-f416-4720-9443-1f3753138a7c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:51.604580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005281-f416-4720-9443-1f3753138a7c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:51.604679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005281-f416-4720-9443-1f3753138a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:51.623273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-04005281-f416-4720-9443-1f3753138a7c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:51.624462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-04005281-f416-4720-9443-1f3753138a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:51.690969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 546/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:51.966983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e86460-0d60-4a1f-b713-3a0f968e733c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:52.064244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e86460-0d60-4a1f-b713-3a0f968e733c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:52.083145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:52.096829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:52.097433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:52.106992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:52.106992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:52.106992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:52.106992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8835c0-90b4-40fe-962d-421c5f9e4366 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:52.106992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 563/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:53:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:52.108477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04005281-f416-4720-9443-1f3753138a7c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:52.204302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04005281-f416-4720-9443-1f3753138a7c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:52.224064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:52.237989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:52.238578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:52.238856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:52.239148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:52.239818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:52.241351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4629c-3ae4-4a31-a8cc-3373407e8971 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:52.312927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 547/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:53:52 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:53:54.232560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b4b11-a505-42c0-b999-fd80d907527b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:54.233095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b4b11-a505-42c0-b999-fd80d907527b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:54.233095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b4b11-a505-42c0-b999-fd80d907527b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:54.234418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 564/1111] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:54.239586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:54.240010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:54.240275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:54.258315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:54.259535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:54.324765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 548/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:54.593628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b4b11-a505-42c0-b999-fd80d907527b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:54.691009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b4b11-a505-42c0-b999-fd80d907527b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:54.724605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:54.725196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:54.725467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:54.725733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] PATH_INFO: `/v3/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:54.727098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] RBAC: Authorizing `identity:get_role(role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:54.728460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:54.729740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 565/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:54 2022] GET /identity/v3/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:54.735484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:54.831640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694ebc-737b-499c-8f91-eeada5fa0b67 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:54.839466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:54.864877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:54.865487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:54.865767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:54.866038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] PATH_INFO: `/v3/users/dca80c7f524440c29aad893407ff579a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:54.880043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] RBAC: Authorizing `identity:get_user(user_id=dca80c7f524440c29aad893407ff579a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:54.882187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:54.883818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 549/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:54 2022] GET /identity/v3/users/dca80c7f524440c29aad893407ff579a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:54.889699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:54.945948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:54.988656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c18a3-d3e0-4961-98d6-b76554a89577 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:55.023410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:55.024035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:55.024306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:55.024571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:55.091900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:55.093264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:55.147714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:55.190825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 566/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:53:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:53:55.198073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:55.292505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953689a-dbef-432c-9ddd-12c9f59bb058 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:55.326984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:55.327627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:55.327907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:55.328181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:55.329277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:55.330702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c33f1a-39e3-409d-aab2-e4aa55c66ff2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:55.332353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 550/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:53:55 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:55.338499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:55.439982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3857618-20d7-4c61-81b7-dafa8e721bed None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:55.472386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:55.472976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:55.473255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:55.473527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:55.474903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d, user.id=dca80c7f524440c29aad893407ff579a) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:53:55.475350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:55.476789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026aeb5-2e58-45d6-8ed9-20ffec3a5822 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:55.482083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 567/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:53:55 2022] GET /identity/v3/role_assignments?user.id=dca80c7f524440c29aad893407ff579a&scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:57.400881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fa9d-c0f7-41d5-8887-4bce44f11123 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:57.401449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fa9d-c0f7-41d5-8887-4bce44f11123 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:57.401449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6fa9d-c0f7-41d5-8887-4bce44f11123 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:57.402766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 551/1118] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:53:57.407917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:57.408298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:57.408469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:57.426365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:53:57.428286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:53:57.442006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:57.493409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 568/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:53:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:53:57.759679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fa9d-c0f7-41d5-8887-4bce44f11123 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:57.790155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:57.858603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6fa9d-c0f7-41d5-8887-4bce44f11123 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:57.891785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:57.892398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:57.892676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:57.892947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] PATH_INFO: `/v3/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:57.894323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] RBAC: Authorizing `identity:get_role(role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:57.895714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:57.897051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 552/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:57 2022] GET /identity/v3/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:57.902797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:57.998396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb54f00-8ac1-422d-b232-e5eabb55c38b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:58.030727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:58.031369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:58.031644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:58.031912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] PATH_INFO: `/v3/users/dca80c7f524440c29aad893407ff579a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:58.033011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] RBAC: Authorizing `identity:get_user(user_id=dca80c7f524440c29aad893407ff579a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:58.034427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:58.035995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 569/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:53:57 2022] GET /identity/v3/users/dca80c7f524440c29aad893407ff579a => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:58.042548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:58.138314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:58.141216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc7bf-bd18-43b7-81ea-70c565bcf7e4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:58.176090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:58.176705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:58.176984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:58.177282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:58.178669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:58.180208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd08e4-d8dd-4475-aa38-7df962767d37 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:58.181827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 553/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:53:58 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:53:58.187909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:53:58.210602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:53:58.290854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662f010-e969-404e-87b1-45fecc986071 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:53:58.323928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:53:58.324567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:53:58.324849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:53:58.325118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d/users/dca80c7f524440c29aad893407ff579a/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:53:58.327239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3061a4ceb35b4e8b85954fc4adf5b83d, user_id=dca80c7f524440c29aad893407ff579a, role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:53:58.328659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c59bb8-c045-4717-972e-85eed9b296c1 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:53:58.421309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 570/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:53:58 2022] PUT /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d/users/dca80c7f524440c29aad893407ff579a/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:54:00.320413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d9be0-0d88-41b5-9fa8-c3b72d1c0b4c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:00.320945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d9be0-0d88-41b5-9fa8-c3b72d1c0b4c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:00.320945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d9be0-0d88-41b5-9fa8-c3b72d1c0b4c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:00.322278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 554/1124] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:00.327379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:00.327778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:00.327911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:00.345711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:00.346884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:00.411021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 571/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:54:00.688451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d9be0-0d88-41b5-9fa8-c3b72d1c0b4c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:00.786187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d9be0-0d88-41b5-9fa8-c3b72d1c0b4c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:00.820828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:00.821434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:00.821716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:00.821991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] PATH_INFO: `/v3/roles/d923b603652944b6b7ef0f08aafe1ae6` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:00.823404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] RBAC: Authorizing `identity:get_role(role_id=d923b603652944b6b7ef0f08aafe1ae6)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:00.824779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:00.826084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 555/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:00 2022] GET /identity/v3/roles/d923b603652944b6b7ef0f08aafe1ae6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:54:00.831815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:00.842060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:00.929527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965791a-caa6-4586-9a8c-e3a70f572be7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:00.962333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:00.962888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:00.963195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:00.963470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] PATH_INFO: `/v3/users/dca80c7f524440c29aad893407ff579a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:00.964576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] RBAC: Authorizing `identity:get_user(user_id=dca80c7f524440c29aad893407ff579a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:00.966003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:00.967597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 572/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:00 2022] GET /identity/v3/users/dca80c7f524440c29aad893407ff579a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:00.973364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:01.007994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:01.071287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf123d4-98b7-421a-b04b-f51babaed98e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:01.105682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:01.106268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:01.106548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:01.106821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:01.171553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:01.173011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:01.204631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:01.267898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 556/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:01.274604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:01.352380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:01.375219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb1aff-c6fb-4cd0-b7d2-6a36cd26f369 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:01.408628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:01.409240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:01.409518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:01.409789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] PATH_INFO: `/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:01.410887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] RBAC: Authorizing `identity:get_project(project_id=3061a4ceb35b4e8b85954fc4adf5b83d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:01.412904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624afd8b-0f89-4197-b805-cfaefbf737bf None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:01.414504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 573/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:01 2022] GET /identity/v3/projects/3061a4ceb35b4e8b85954fc4adf5b83d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:01.420686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:01.517234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2a407-901d-447d-bf28-5b971914996c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:01.550286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:01.550875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:01.551175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:01.551451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:01.552157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] RBAC: Adding query filter params (role.id=d923b603652944b6b7ef0f08aafe1ae6, scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d, user.id=dca80c7f524440c29aad893407ff579a) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:01.553162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:01.554580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3a421-8e91-47e3-83d1-10b68e677d19 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:01.557445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:01.560824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 557/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:54:01 2022] GET /identity/v3/role_assignments?user.id=dca80c7f524440c29aad893407ff579a&scope.project.id=3061a4ceb35b4e8b85954fc4adf5b83d&role.id=d923b603652944b6b7ef0f08aafe1ae6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:03.446191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f62a6-3790-4989-99aa-ac502b971acd None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:03.446752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f62a6-3790-4989-99aa-ac502b971acd None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:03.446752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f62a6-3790-4989-99aa-ac502b971acd None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:03.448102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 574/1131] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:03.453166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:03.453594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:03.453685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:03.472024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:03.473179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:03.537457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 558/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:03.803411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f62a6-3790-4989-99aa-ac502b971acd None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:03.903850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f62a6-3790-4989-99aa-ac502b971acd None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:03.935962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:03.936580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:03.936857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:03.937126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:03.938281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:03.939736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b0d95-2fc0-4844-b992-e395b2fa8888 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:03.941273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 575/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:54:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 08:54:03.947771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:04.043229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc87949-1e8e-4ba5-9376-8dad6c33ff3c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:04.071602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:04.077135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:04.077722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:04.078026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:04.078295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:04.078923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:04.080371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f5501-f9e7-4074-ac4e-8340ce60878e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:04.166825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 559/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:54:03 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:54:06.094505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235a99b-69c1-4cea-a686-30926602b7c4 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:06.095032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235a99b-69c1-4cea-a686-30926602b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:06.095032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235a99b-69c1-4cea-a686-30926602b7c4 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:06.096409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 576/1135] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:06.101857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:06.102262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:06.102400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:06.120329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:06.121490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:06.186175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 560/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:06.458051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235a99b-69c1-4cea-a686-30926602b7c4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:06.473197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:06.559878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235a99b-69c1-4cea-a686-30926602b7c4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:06.593295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:06.593905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:06.594178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:06.594443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:06.595546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:06.596946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21457d-8a2e-46da-8609-1e2f004960fb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:06.598491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 577/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:54:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:06.604507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:06.700655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b55eb-5705-4a5f-aa41-ce9612a8f500 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:06.714040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:06.734993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:06.735667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:06.736153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:06.736637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:06.737653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:06.739150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0fb9e-ec19-4199-8e4f-d465879c73bd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:06.811032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 561/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:54:06 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:54:08.746518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90069ac-ec5a-4d5b-8a2d-c7162a9e75d9 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:08.747078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90069ac-ec5a-4d5b-8a2d-c7162a9e75d9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:08.747078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90069ac-ec5a-4d5b-8a2d-c7162a9e75d9 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:08.748401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 578/1139] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:08.753465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:08.753844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:08.754020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:08.772618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:08.773799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:08.838789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 562/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:09.108084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90069ac-ec5a-4d5b-8a2d-c7162a9e75d9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:09.187045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:09.205615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90069ac-ec5a-4d5b-8a2d-c7162a9e75d9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:09.239098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:09.239727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:09.240124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:09.240395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:09.250302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:09.254999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:09.256109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:09.257034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 579/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:09.262177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:09.355906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:09.359602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630335-9410-4f1b-ac66-ac039bfcf748 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:09.392734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:09.393320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:09.393598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:09.393869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:09.394519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:09.394819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:09.396728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:09.404664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 563/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:09.410313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:09.506054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a579-0c0a-4961-b631-ddb3dd5d6a78 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:09.538982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:09.539608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:09.539881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:09.540147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] PATH_INFO: `/v3/users/203cadadd9ca4973a164e77708e2331a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:09.549940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:09.553564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] RBAC: Authorizing `identity:get_user(user_id=203cadadd9ca4973a164e77708e2331a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:09.554834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:09.556469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 580/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/users/203cadadd9ca4973a164e77708e2331a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:09.562361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:09.660662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957d673-9304-416b-a0dc-2d572461b7fb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:09.692170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:09.692756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:09.693827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:09.694115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:09.720491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:09.759044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:09.760485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:09.853942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 564/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:09.859873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:09.956363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebe026-4de5-4528-bea6-827fea008b83 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:09.988853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:09.989459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:09.989736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:09.990566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] PATH_INFO: `/v3/projects/0b751f56c5b34f988fb8617f68faab2e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:09.991695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] RBAC: Authorizing `identity:get_project(project_id=0b751f56c5b34f988fb8617f68faab2e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:09.993102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774795e0-5b69-4151-98e4-f4935155919f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:09.994949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 581/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/projects/0b751f56c5b34f988fb8617f68faab2e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:10.000934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:10.057463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:10.095270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ab8ed-48e3-4415-b7d3-f8bbaabe29d1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:10.128253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:10.128831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:10.129106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:10.129373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:10.130075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=0b751f56c5b34f988fb8617f68faab2e, user.id=203cadadd9ca4973a164e77708e2331a) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:10.130372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:10.132417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864602ca-1e06-4b0b-97dc-71914d28c933 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:10.137512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 565/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:54:09 2022] GET /identity/v3/role_assignments?user.id=203cadadd9ca4973a164e77708e2331a&scope.project.id=0b751f56c5b34f988fb8617f68faab2e&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:11.981486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828f0f-da32-4d9c-8193-645ebe956e5c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:11.981990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828f0f-da32-4d9c-8193-645ebe956e5c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:11.981990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828f0f-da32-4d9c-8193-645ebe956e5c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:11.983434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 582/1147] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:11.988423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:11.988821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:11.988960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:12.007585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:12.008749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:12.072330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 566/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:12.335201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828f0f-da32-4d9c-8193-645ebe956e5c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:12.400255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:12.433999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828f0f-da32-4d9c-8193-645ebe956e5c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:12.467288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:12.467903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:12.468183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:12.468456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:12.478980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:12.483813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:12.484928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:12.486059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 583/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:12.490920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:12.544444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:12.588559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d890e6-6566-4a03-97ed-5b3913f0def7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:12.621002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:12.621592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:12.621867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:12.622136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:12.622785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:12.623080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:12.624461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:12.632844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 567/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:12.638603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:12.734460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdba161-1d8b-4a75-a0aa-d841de201c87 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:12.760034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:12.768402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:12.769013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:12.780556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:12.780556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] PATH_INFO: `/v3/users/203cadadd9ca4973a164e77708e2331a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:12.780556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] RBAC: Authorizing `identity:get_user(user_id=203cadadd9ca4973a164e77708e2331a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:12.780556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:12.780556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 584/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:12 2022] GET /identity/v3/users/203cadadd9ca4973a164e77708e2331a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:12.780977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:12.880492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567bff3-9e13-46b9-a5e7-4f264095f1c8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:12.896058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:12.914330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:12.914938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:12.915249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:12.915525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] PATH_INFO: `/v3/projects/0b751f56c5b34f988fb8617f68faab2e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:12.916655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] RBAC: Authorizing `identity:get_project(project_id=0b751f56c5b34f988fb8617f68faab2e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:12.918078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cf8cc-1000-471f-bf3d-a4fefdb50721 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:12.919704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 568/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:12 2022] GET /identity/v3/projects/0b751f56c5b34f988fb8617f68faab2e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:12.925567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:13.020759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e9e-6ce9-44e0-8ef5-0b985cbf07e7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:13.054048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:13.054680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:13.054956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:13.055255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] PATH_INFO: `/v3/projects/0b751f56c5b34f988fb8617f68faab2e/users/203cadadd9ca4973a164e77708e2331a/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:13.057115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b751f56c5b34f988fb8617f68faab2e, user_id=203cadadd9ca4973a164e77708e2331a, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:13.058527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4a7fb-b4cd-4193-b2ad-624526c20f79 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:13.159247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 585/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:54:12 2022] PUT /identity/v3/projects/0b751f56c5b34f988fb8617f68faab2e/users/203cadadd9ca4973a164e77708e2331a/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:54:15.012259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8419-315e-4b2e-824f-2dd73ac7a421 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.012783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8419-315e-4b2e-824f-2dd73ac7a421 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.012783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8419-315e-4b2e-824f-2dd73ac7a421 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:15.014052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 569/1154] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:15.019232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.019613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.019776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:15.038328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:15.039533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:15.087729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:15.104978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 586/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:54:15.374489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d8419-315e-4b2e-824f-2dd73ac7a421 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:15.472598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d8419-315e-4b2e-824f-2dd73ac7a421 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:15.505117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:15.505751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.506268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.506555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:15.516626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:15.522089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:15.523218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:54:15.524542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 570/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:15.529033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:15.627833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da31563-bf34-46e3-b158-53ea67ac334d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:15.661128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:15.661720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.662016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.662288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:15.663075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:15.663408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:15.664767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:15.672514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 587/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:15.678375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:15.722752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:15.775899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc568-f63d-4d4b-b2c7-9f6cf1d9c3a9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:15.807806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:15.808413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.808692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.808966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] PATH_INFO: `/v3/users/203cadadd9ca4973a164e77708e2331a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:15.810062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] RBAC: Authorizing `identity:get_user(user_id=203cadadd9ca4973a164e77708e2331a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:15.811528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:15.813082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 571/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:15 2022] GET /identity/v3/users/203cadadd9ca4973a164e77708e2331a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:15.819372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:15.866362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:15.917530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6140-0178-4e41-99bd-f4935a94ee9a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:15.949649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:15.950236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:15.950510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:15.950778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:16.017047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:16.018479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:16.062898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:16.115400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 588/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:16.121704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:16.219979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5119c7-da17-4a91-96fa-aa3fa1d328a4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:16.232994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:16.253971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:16.254587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:16.254870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:16.255176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] PATH_INFO: `/v3/projects/0b751f56c5b34f988fb8617f68faab2e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:16.256999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] RBAC: Authorizing `identity:get_project(project_id=0b751f56c5b34f988fb8617f68faab2e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:16.258434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353a6bb-ca78-4a44-a594-fc4cf4c94781 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:16.260252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 572/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:16 2022] GET /identity/v3/projects/0b751f56c5b34f988fb8617f68faab2e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:16.266437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:16.362135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87cd4d-2c07-4d7d-bc86-a3d5534551ea None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:16.395072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:16.395679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:16.395955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:16.396246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:16.396979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=0b751f56c5b34f988fb8617f68faab2e, user.id=203cadadd9ca4973a164e77708e2331a) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:16.397391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:16.398809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d1daf-22da-44b4-8f7b-4d3fed1d2853 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:16.404582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 589/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:54:16 2022] GET /identity/v3/role_assignments?user.id=203cadadd9ca4973a164e77708e2331a&scope.project.id=0b751f56c5b34f988fb8617f68faab2e&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:18.335395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e24f-a926-46ae-bd89-01e68def3afe None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:18.335927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e24f-a926-46ae-bd89-01e68def3afe None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:18.335927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e24f-a926-46ae-bd89-01e68def3afe None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:18.337266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 573/1162] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:18.342314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:18.342764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:18.342865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:18.361188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:18.362337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:18.365041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:18.426506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 590/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:18.696149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e24f-a926-46ae-bd89-01e68def3afe None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:18.794840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e24f-a926-46ae-bd89-01e68def3afe None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:18.826598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:18.827224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:18.827501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:18.827769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:18.873411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:18.892214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:18.893650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:18.988523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 574/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:18.995144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:19.089438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17ace4-d89d-42dd-a5cc-5c140c536e89 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:19.121899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:19.122503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:19.122774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:19.123037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:19.129393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:19.131444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:19.133849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-e114f668-2d0c-4521-b02a-9b8a90f9fc68 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:19.142810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:19.148579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:19.149656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 591/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:54:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:19.150672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:19.227638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:19.246651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f52d7-d556-4ffd-b1a0-8b5ad7bf1319 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:19.279722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:19.280308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:19.280584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:19.280851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:19.281472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:19.281769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:19.283131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759afa-386c-45f4-8490-64f5e349482a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:19.290050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 575/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 08:54:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:21.160433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da15a-25b7-44a0-bc69-3502d63d3b52 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:21.161020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da15a-25b7-44a0-bc69-3502d63d3b52 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:21.161020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da15a-25b7-44a0-bc69-3502d63d3b52 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:21.162336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 592/1167] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:21.167529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5cc54-bacb-4b56-9b7f-74b7f4955d93 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:21.167971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5cc54-bacb-4b56-9b7f-74b7f4955d93 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:21.168093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5cc54-bacb-4b56-9b7f-74b7f4955d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:21.185931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-14b5cc54-bacb-4b56-9b7f-74b7f4955d93 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:21.187123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-14b5cc54-bacb-4b56-9b7f-74b7f4955d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:21.251939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 576/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:21.520330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394da15a-25b7-44a0-bc69-3502d63d3b52 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:21.619950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394da15a-25b7-44a0-bc69-3502d63d3b52 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:21.651612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:21.652402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:21.652683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:21.652951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:21.653608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:21.654942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607ec3a-61d0-46bf-9576-ff900a32e891 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:21.744218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 593/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu Sep 1 08:54:21 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:54:23.685540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b0ef-1cce-4991-8000-11f60e634f2f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:23.686088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b0ef-1cce-4991-8000-11f60e634f2f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:23.686088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b0ef-1cce-4991-8000-11f60e634f2f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:23.687462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 577/1170] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:23.692472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:23.692878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:23.693014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:23.711259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:23.712410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:23.765994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:23.776662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 594/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:24.055258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b0ef-1cce-4991-8000-11f60e634f2f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:24.132711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:24.157380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b0ef-1cce-4991-8000-11f60e634f2f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:24.191499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:24.192106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:24.192382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:24.192652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] PATH_INFO: `/v3/domains/533988dbbe084ac4a87f85220dd7a18d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:24.202928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=533988dbbe084ac4a87f85220dd7a18d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:24.204397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7e037-12c2-40e6-bdea-095e93f978fa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:24.205956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 578/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Sep 1 08:54:24 2022] GET /identity/v3/domains/533988dbbe084ac4a87f85220dd7a18d => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:24.212270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:24.311233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c1fe4-1067-4980-b25f-0624deb112de None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:24.343582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:24.344173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:24.344449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:24.344716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:24.345346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:24.346764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3dcd3-c09d-4264-a201-6d74065ba912 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:24.438255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 595/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:54:24 2022] POST /identity/v3/projects => generated 361 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:54:26.373514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60003b3-04e3-4207-adbf-925d48e0c466 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:26.374085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60003b3-04e3-4207-adbf-925d48e0c466 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:26.374085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60003b3-04e3-4207-adbf-925d48e0c466 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:26.375439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 579/1174] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:26.380730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:26.381183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:26.381420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:26.399319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:26.400478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:26.434604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:26.464732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 596/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:26.732669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60003b3-04e3-4207-adbf-925d48e0c466 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:26.831469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60003b3-04e3-4207-adbf-925d48e0c466 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:26.863939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:26.864543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:26.864824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:26.865096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] PATH_INFO: `/v3/domains/533988dbbe084ac4a87f85220dd7a18d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:26.866234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=533988dbbe084ac4a87f85220dd7a18d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:26.867686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadb410-9fba-464b-ad2a-eede0ad9d0e5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:26.869218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 580/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Sep 1 08:54:26 2022] GET /identity/v3/domains/533988dbbe084ac4a87f85220dd7a18d => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:26.875097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:26.969451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619bf12-0fa9-4a4e-b403-d54c7b116c91 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:27.001687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:27.002207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:27.002480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:27.002744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:27.003408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:27.004814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67614-d42c-48ed-af9b-18cf46e9b54b None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:27.108782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 597/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:54:26 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 08:54:29.065118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055c30-98b5-4db8-bcc5-0b5744f955f0 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:29.065656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055c30-98b5-4db8-bcc5-0b5744f955f0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:29.065656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055c30-98b5-4db8-bcc5-0b5744f955f0 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:29.067008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 581/1178] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:29.072801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:29.073316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:29.073453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:29.091263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:29.092438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:29.115092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:29.158270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 598/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:29.425363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055c30-98b5-4db8-bcc5-0b5744f955f0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:29.428887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:29.526650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055c30-98b5-4db8-bcc5-0b5744f955f0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:29.559409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:29.560030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:29.560309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:29.560581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:29.570974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:29.575759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:29.576929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:54:29.577895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 582/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:29.582802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:29.678246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de9ae9-0192-444d-8e69-54c0ff5f8821 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:29.710858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:29.711465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:29.711742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:29.712011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:29.712654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:29.712951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:29.714296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:29.721995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 599/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:29.728340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:29.788468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:29.825975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5631b01-f397-4dc3-b474-7d7a43427a8e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:29.858656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:29.859301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:29.859586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:29.859861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] PATH_INFO: `/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:29.872926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] RBAC: Authorizing `identity:get_user(user_id=ab38d1633b1247aca3d0f5e42eeadfe5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:29.874361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:29.875978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 583/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:29 2022] GET /identity/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:29.882324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:29.904520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:29.980026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b641e0-2c4c-42a2-9548-b7578e6cc1ef None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:30.013078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:30.013662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:30.013934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:30.014198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:30.078471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:30.079934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:30.099911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:30.176017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 600/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:30.182145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:30.280208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f22-40a6-404f-9998-30eeea4b6cbb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:30.299724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:30.313724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:30.314366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:30.314647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:30.314917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] PATH_INFO: `/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:30.316050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] RBAC: Authorizing `identity:get_project(project_id=a56d0e79d24b40ab99b04e2f32cd40f7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:30.317465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d9553-403b-4959-981b-07d03c17ae12 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:30.319915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 584/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:30 2022] GET /identity/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:30.325909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:30.421307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e2cb2-5b1d-401b-8956-0112b9014f6a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:30.440517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:30.454352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:30.454940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:30.455245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:30.455517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:30.456225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=a56d0e79d24b40ab99b04e2f32cd40f7, user.id=ab38d1633b1247aca3d0f5e42eeadfe5) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:30.456528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:30.457942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422f5e-0f6d-4c95-9287-07d5cfca294f None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:30.463066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 601/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:54:30 2022] GET /identity/v3/role_assignments?user.id=ab38d1633b1247aca3d0f5e42eeadfe5&scope.project.id=a56d0e79d24b40ab99b04e2f32cd40f7&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:32.382233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569238-6f9e-4d30-97ec-59b99a1753b4 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:32.382794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569238-6f9e-4d30-97ec-59b99a1753b4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:32.382794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569238-6f9e-4d30-97ec-59b99a1753b4 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:32.384155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 585/1186] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:32.389323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:32.389712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:32.389873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:32.408821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:32.410005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:32.475901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 602/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:32.751406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569238-6f9e-4d30-97ec-59b99a1753b4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:32.850145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569238-6f9e-4d30-97ec-59b99a1753b4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:32.883444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:32.884047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:32.884328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:32.884600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:32.894342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:32.899210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:32.900308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:54:32.901206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 586/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:32.906161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:33.002266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ff1f-c902-4c3a-97c4-24ce51c16a09 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:33.015477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:33.034129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:33.034720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:33.034996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:33.035299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:33.035946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:33.036245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:33.038135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:33.045925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 603/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:33.051685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:33.100387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:33.149013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197c923-cfe4-4814-a495-7709d27521e1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:33.181885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:33.182485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:33.182761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:33.183033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] PATH_INFO: `/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:33.184144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] RBAC: Authorizing `identity:get_user(user_id=ab38d1633b1247aca3d0f5e42eeadfe5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:33.185552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:33.187094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 587/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:33 2022] GET /identity/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:33.192807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:33.290735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3ca92-4cab-4ed2-9c86-d80a9c8b0bff None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:33.322593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:33.323226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:33.323507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:33.323779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] PATH_INFO: `/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:33.324882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] RBAC: Authorizing `identity:get_project(project_id=a56d0e79d24b40ab99b04e2f32cd40f7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:33.326306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd558d-e5ec-4771-b61e-b0aed8a5bef7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:33.327927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 604/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:33 2022] GET /identity/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:33.333630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:33.431583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9ac53-21c6-41cb-a10a-6c25dec5081d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:33.447272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:33.464796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:33.465434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:33.465789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:33.466057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] PATH_INFO: `/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7/users/ab38d1633b1247aca3d0f5e42eeadfe5/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:33.467946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] RBAC: Authorizing `identity:create_grant(project_id=a56d0e79d24b40ab99b04e2f32cd40f7, user_id=ab38d1633b1247aca3d0f5e42eeadfe5, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:33.469365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4b000-b2ac-4477-b537-1736c0239b92 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:33.535322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 588/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:54:33 2022] PUT /identity/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7/users/ab38d1633b1247aca3d0f5e42eeadfe5/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:54:35.407976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a608-b736-4904-8a04-6839a8db9e24 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:35.408625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a608-b736-4904-8a04-6839a8db9e24 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:35.408625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a608-b736-4904-8a04-6839a8db9e24 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:35.409866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 605/1193] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:54:35.415284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:35.415699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:35.415836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:35.433751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:54:35.434935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:54:35.500642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 589/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:54:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:54:35.765743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a608-b736-4904-8a04-6839a8db9e24 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:35.812857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:35.866307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a608-b736-4904-8a04-6839a8db9e24 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:35.899260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:35.899869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:35.900242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:35.900513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:35.910539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:54:35.915708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:54:35.916862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:54:35.918001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 606/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:54:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:54:35.922849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:36.022691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bc91e-fd3a-4910-a4f0-cb96fa246bd2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:36.025949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:36.051367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:36.051956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:36.052231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:36.052500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:36.053258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:36.053657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:36.055017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:36.062863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 590/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:54:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:36.069199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:36.169322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fe518-3f49-48db-a54a-c9b5823506d6 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:36.171648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:36.202608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:36.203394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:36.203675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:36.203946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] PATH_INFO: `/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:36.205054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] RBAC: Authorizing `identity:get_user(user_id=ab38d1633b1247aca3d0f5e42eeadfe5)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:36.206482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:36.208080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 607/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 08:54:36 2022] GET /identity/v3/users/ab38d1633b1247aca3d0f5e42eeadfe5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:36.214536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:36.311828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fa372-1318-4727-ad71-400113c33bd3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:36.343936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37065941-2c56-41bb-996d-247300734840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:36.344528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065941-2c56-41bb-996d-247300734840 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:36.344809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065941-2c56-41bb-996d-247300734840 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:36.345083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065941-2c56-41bb-996d-247300734840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:36.378190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:36.410172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37065941-2c56-41bb-996d-247300734840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:36.411672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37065941-2c56-41bb-996d-247300734840 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:36.506483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 591/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:54:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:54:36.512371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:36.609504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70a6c4-488c-42cb-b21a-f771faf2a564 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:36.642512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:36.643144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:36.643428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:36.643701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] PATH_INFO: `/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:36.644801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] RBAC: Authorizing `identity:get_project(project_id=a56d0e79d24b40ab99b04e2f32cd40f7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:36.646217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac80f59-92bb-4610-94c4-5ddc4096b938 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:36.648701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 608/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 08:54:36 2022] GET /identity/v3/projects/a56d0e79d24b40ab99b04e2f32cd40f7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:36.654821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37065941-2c56-41bb-996d-247300734840 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:54:36.718672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:36.750023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37065941-2c56-41bb-996d-247300734840 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:54:36.783656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:54:36.784241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:54:36.784516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:54:36.784784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:54:36.785481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=a56d0e79d24b40ab99b04e2f32cd40f7, user.id=ab38d1633b1247aca3d0f5e42eeadfe5) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:54:36.785779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:54:36.787240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298c54-4a2e-4251-a7c5-b670bfe02150 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:54:36.792802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 592/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:54:36 2022] GET /identity/v3/role_assignments?user.id=ab38d1633b1247aca3d0f5e42eeadfe5&scope.project.id=a56d0e79d24b40ab99b04e2f32cd40f7&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:54:46.638430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=80 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:54:46.790292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=34 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:48.382326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be497-200d-442a-94a4-c5840d30bd2e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:48.382917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be497-200d-442a-94a4-c5840d30bd2e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:48.382917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be497-200d-442a-94a4-c5840d30bd2e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:48.384323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 609/1201] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:55:48.389879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:48.390303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:48.390412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:48.408477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:55:48.409662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:55:48.476893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 593/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:55:48.749878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be497-200d-442a-94a4-c5840d30bd2e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:48.848155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be497-200d-442a-94a4-c5840d30bd2e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:48.882454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:48.883072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:48.883385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:48.883656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:48.893612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:48.895031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:55:48.909710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:48.918959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:55:48.920063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:55:48.920063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:48.920063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:55:48.920063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 610/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:48.920321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:49.015397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db31ac4-81e5-48fe-88df-eb7cc692974f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:49.043761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:49.049282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:49.049876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:49.050152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:49.050420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:49.051047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:49.051371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:49.052724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4836a4-1a93-43bd-b89f-8ff88de370fa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:49.063693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 594/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:49.069465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:49.122914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:49.170915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a47001-0940-42b3-8d7d-8c6413be6148 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:49.204106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:49.204706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:49.204977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:49.205242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:49.205885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:49.207341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a0715-458b-4339-a0d7-1ccee9bc89b3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:49.294686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 611/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 08:55:49 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:55:51.206616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932759a6-294d-49c0-a5c8-f6f12958ec18 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:51.207227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932759a6-294d-49c0-a5c8-f6f12958ec18 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:51.207336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932759a6-294d-49c0-a5c8-f6f12958ec18 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:51.208664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 595/1206] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:55:51.213600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:51.214522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:51.214688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:51.233077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:55:51.235869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:55:51.300223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 612/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:55:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:55:51.565541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932759a6-294d-49c0-a5c8-f6f12958ec18 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:51.667001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932759a6-294d-49c0-a5c8-f6f12958ec18 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:51.700342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:51.700999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:51.701281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:51.701549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:51.718610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:55:51.723458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:51.724621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:51.725677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 596/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:55:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:51.730940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:51.770606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:51.827453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b493771-8020-47f8-ac50-d6ecb768a01a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:51.862627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:51.863252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:51.863527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:51.863793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:51.864440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:51.864734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:51.866081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:51.874500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 613/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:55:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:51.880327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:51.929076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:51.977770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e46c7d-6c17-44e6-ac9b-d0e2afdadf0c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.011533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.012114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.012400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.012669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.023115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.024573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:55:52.038870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:52.040107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:55:52.041289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:55:52.041289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:52.041289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:52.041289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 597/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:52.046291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:52.143878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:52.146731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afc4a5-f76e-41b5-961b-379df8f13f68 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.179500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.180089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.180364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.180635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.181261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:52.181559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.183552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.194575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 614/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:52.201137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:52.302486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab178c-b0aa-4e0f-a5b8-a2f9e80e94ca None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.304031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:52.337261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.337918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.338195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.338469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.346189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.347665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:55:52.359930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:52.361198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:52.362410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 598/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:52.366978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:52.464958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed931-9f33-4cac-b5a6-634695b19a3c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.498736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.499361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.499635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.499902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.500610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:52.500910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.502323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.519533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 615/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:52.525362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:52.624994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03cae8-958c-4ca8-97a4-2b8f61eeea0f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.657812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.658405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.658678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.659031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.688436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:52.725198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.726593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.824388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 599/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:55:52.830034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:52.849223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:52.927296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95cd0f-3ce7-472c-80f7-bb64d161195e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:52.960817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:52.961425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:52.961696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:52.961961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:52.972426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:52.973869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:55:52.988602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:52.989785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:55:52.990912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:55:52.990912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:52.990912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:55:52.990912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 616/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:52.995605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:53.057734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:53.096528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968a89f-e8fe-4b88-b2c5-3de8b5091dae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:53.130531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:53.132662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:53.132662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:53.132662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:53.132662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:53.132662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:53.133966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:53.146011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 600/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:53.152441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:53.218882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:53.247532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d00aaf-c01f-4cad-a030-7cf2fa229f90 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:53.279688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:53.280296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:53.280569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:53.280836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:53.287069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:53.288521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:55:53.299667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:53.300797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:55:53.301848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 617/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:53.306896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:53.405731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec89133-7b04-4d60-b534-4fdd5950cf2c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:53.427278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:53.438964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:53.439579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:53.439854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:53.440120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:53.440797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:53.441100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:53.442569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865432-1b4c-44bd-a52d-9c6cd8c09108 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:53.454718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 601/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:53.460772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:53.559476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4652-0355-491d-bb34-26695c6624bd None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:53.588091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:53.594085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:53.594667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:53.594944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:53.595245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:53.595952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=97133f7b0f5b4320a41486752de09440) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:53.596255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:53.597725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95393e-fced-4b5f-8261-6e0d7d284752 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:53.603019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 618/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:55:53 2022] GET /identity/v3/role_assignments?user.id=97133f7b0f5b4320a41486752de09440&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:55.482503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed193-5b73-439e-ac1f-a622eef39f09 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:55.483060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed193-5b73-439e-ac1f-a622eef39f09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:55.483060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ed193-5b73-439e-ac1f-a622eef39f09 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:55.484417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 602/1220] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:55:55.489733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:55.490163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:55.490261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:55.508961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:55:55.510177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:55:55.574719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 619/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:55:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 08:55:55.856772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ed193-5b73-439e-ac1f-a622eef39f09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:55.959615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ed193-5b73-439e-ac1f-a622eef39f09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:55.995523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:55.996539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:55.996827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:55.997099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.007220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:55:56.013082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:56.014084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:56.015185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 603/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:56.020597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.118622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13dd05-2154-409b-870b-cffe847b84f0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.142330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:56.153396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.153992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.154265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.154533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.155207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] RBAC: Adding query filter params (name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:56.155506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.156900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.164720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 620/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:56.170673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.213913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:56.268417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e68-798c-4d51-a90c-5b8a9a903b63 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.301601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.302206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.302483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.302750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.312675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.314321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:55:56.329347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:56.330595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:55:56.335759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:55:56.335759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:56.335759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:56.335759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 604/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:56.336839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.438063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e16ac-7f73-4b0a-af0d-0ac9773a4703 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.471313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.471912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.472191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.472462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.473092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:56.473392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.474765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.486344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 621/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:56.492133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.586259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:56.592887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0c950-03ae-46bf-84d5-51e61bc7385d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.625610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.626221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.626502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.626774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.633953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.635433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:55:56.647779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:56.650739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:56.651819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 605/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:56.654846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.702604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:56.752370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de98777-4b05-49a9-bc7c-d3cb88b66572 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.787017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.787631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.787903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.788164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.788872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:56.789170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.790579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.807931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 622/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:56.814134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:56.913094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2386bb-e5a5-4a22-8de6-bb47d0eaba7d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:56.946994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:56.947633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:56.947907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:56.948177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:56.966998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:56.966998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:55:56.976565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:56.977706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:55:56.978855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:55:56.978855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:55:56.978855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:56.978855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 606/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:56.985582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:57.088760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8906-c29d-4142-89ca-bff66fb525ec None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:57.091482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:57.122762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:57.123643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:57.123915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:57.124182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:57.124837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:57.125131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:57.126476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:57.137263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 623/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:57.143162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:57.153523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:57.243528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecfcf7-37fe-4f22-afd9-ad47e7961e4e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:57.275969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:57.277155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:57.277969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:57.278466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:57.285829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:57.288984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:55:57.299613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:55:57.300703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:55:57.301844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 607/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:55:57.306697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:57.405528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb72f2-4883-4aac-8e96-febf327dd1a3 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:57.439407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:57.440001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:57.440278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:57.440548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:57.441227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:55:57.441526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:57.442948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace45ef-841a-4432-8d23-f140e18a27da None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:57.455094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 624/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:55:57.461284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:57.525977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:55:57.559511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da3d9-63c1-4fb8-82ff-7dd6f61f3386 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:55:57.593327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:55:57.594002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:57.594283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:57.594552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/97133f7b0f5b4320a41486752de09440/roles/304d887d3553492bbb31f573a7952646` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:57.608591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=97133f7b0f5b4320a41486752de09440, role_id=304d887d3553492bbb31f573a7952646)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:55:57.610037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5495-16f5-49e0-ba8c-5cf10a2cb4dc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:55:57.672599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 608/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:55:57 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/97133f7b0f5b4320a41486752de09440/roles/304d887d3553492bbb31f573a7952646 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:55:59.558008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742249a-f627-4944-a25c-3d746772671e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:59.558787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742249a-f627-4944-a25c-3d746772671e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:59.558787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742249a-f627-4944-a25c-3d746772671e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:59.559995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 625/1233] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:55:59.565435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e761c-f285-4857-801b-c742cea0b437 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:55:59.565873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e761c-f285-4857-801b-c742cea0b437 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:55:59.565998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e761c-f285-4857-801b-c742cea0b437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:55:59.586121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-037e761c-f285-4857-801b-c742cea0b437 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:55:59.587530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-037e761c-f285-4857-801b-c742cea0b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:55:59.659708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 609/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:55:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:55:59.944038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742249a-f627-4944-a25c-3d746772671e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:55:59.972016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:00.057519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742249a-f627-4944-a25c-3d746772671e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.095786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.096611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.096887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.097153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.107565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:56:00.112362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:00.113485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:00.114536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 626/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 08:55:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:00.120082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e761c-f285-4857-801b-c742cea0b437 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:00.169487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:00.228282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e761c-f285-4857-801b-c742cea0b437 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.262179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.262770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.263044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.263333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.263982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:00.264274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:00.265632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:00.273335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 610/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:00.279798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:00.364390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:00.380295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b29614-4d7a-4aca-9c59-5a4fe5237343 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.412964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.413588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.413863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.414175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.424556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:00.428347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:00.441243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:00.442468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:00.443759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:00.443759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:00.443759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:00.443759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 627/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:00.448544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:00.547861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2163ea-d88a-4f01-ac6b-23072eb0cb82 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.557117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:00.582350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.583024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.583234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.583503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.584124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:00.584420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:00.585786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:00.597175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 611/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:00.602974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:00.703220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade511f3-78cf-4f00-9a14-6fb9fcf53faa None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.735728 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.736336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.736607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.736872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.739567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:00.744468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:00.746757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:56:00.757812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:00.758951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:00.760268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 628/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:00.764910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:00.866405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c7d9-e9a7-4910-86c2-37e13b4afccd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:00.900833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:00.901433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:00.902271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:00.902609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:00.903363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:00.903714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:00.905152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:00.922473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 612/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:00.928308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.026362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466608e-b422-47c1-a06c-fa53c40cd16e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:01.062468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:01.063055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:01.063357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:01.063625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:01.124378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:01.130681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:01.132156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:01.229013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 629/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:56:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:56:01.235683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.262965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:01.330952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f574c2f-6e05-4f57-bbab-1337c0b7f290 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:01.364300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:01.364920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:01.365202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:01.365474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:01.375616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:01.377004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:01.392180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:01.400234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:01.401170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:01.401170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:01.401170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:01.401170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 613/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:01.401415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.493664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:01.499688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce232-2b61-4e0e-9817-09910bc097eb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:01.533430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:01.534063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:01.534337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:01.534602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:01.535257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:01.535551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:01.536897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:01.547660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 630/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:01.553496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.633216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:01.653776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646393b5-d28a-49f4-b728-93bc97d7e6c2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:01.686011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:01.686610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:01.686883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:01.687177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:01.693464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:01.694887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:56:01.706580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:01.707667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:01.708671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 614/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:01.713700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.811861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9041f-dd72-42e1-a56f-495c3e51083c None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:01.845365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:01.845959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:01.846234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:01.846500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:01.847230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:01.847534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:01.849103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632893f5-7735-4e09-aec1-e59519aa9c1e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:01.861195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 631/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:01.867558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:01.968454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af538b2-2e48-4f5e-90ef-92fedd04a876 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:02.001060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:02.001635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:02.001908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:02.002347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:02.002937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] RBAC: Adding query filter params (role.id=304d887d3553492bbb31f573a7952646, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=97133f7b0f5b4320a41486752de09440) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:02.003264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:02.004672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8293d4f-3a64-4b27-b139-cafe07579946 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:02.010482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 615/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:56:01 2022] GET /identity/v3/role_assignments?user.id=97133f7b0f5b4320a41486752de09440&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=304d887d3553492bbb31f573a7952646 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:03.975813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5c93-4491-48af-83de-d2f9b5ad9e00 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:03.976317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5c93-4491-48af-83de-d2f9b5ad9e00 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:03.976317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5c93-4491-48af-83de-d2f9b5ad9e00 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:03.977591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 632/1247] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:03.983417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:03.983804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:03.983989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:03.990217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:04.002513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:04.008061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:04.067557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 616/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:04.335949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd5c93-4491-48af-83de-d2f9b5ad9e00 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:04.354901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:04.438295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd5c93-4491-48af-83de-d2f9b5ad9e00 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:04.472089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:04.472702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:04.472977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:04.473244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:04.483400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:56:04.488191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:04.489179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:04.490126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 633/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:04.495633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:04.595892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746779-0f3a-4df7-97b7-f8ac1141e614 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:04.625443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:04.630177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:04.630781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:04.631065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:04.631369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:04.632029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:04.632359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:04.633784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:04.642482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 617/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:04.648221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:04.724510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:04.746999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084f104-7e1a-4ba3-b5e9-5c1377bb1793 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:04.781292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:04.781944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:04.782240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:04.782517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:04.792314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:04.793745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:04.807972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:04.811095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:04.812304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:04.812304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:04.812304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:04.812304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 634/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:04.815733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:04.917331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6a92-5329-48cf-93aa-9352dc84620e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:04.951518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:04.952114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:04.952395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:04.952666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:04.953299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:04.953628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:04.955012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:04.966662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 618/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:04.972447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:05.071558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3824-ea1b-454e-ba4c-282b55eabcb9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:05.096843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:05.104513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:05.106017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:05.106392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:05.106662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] PATH_INFO: `/v3/users/placement` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:05.114648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:05.116205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:56:05.128148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:05.129319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:05.130692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 635/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:05.135215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:05.191863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:05.234977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f415c1-96be-4a53-a65e-e0034db03294 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:05.269963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:05.270566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:05.270839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:05.271125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:05.271850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:05.272148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:05.273578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:05.290005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 619/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:56:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:05.296754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:05.396247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af936212-4d24-4851-a3f4-adb45b5ee3db None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:05.429808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:05.430397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:05.430670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:05.430936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:05.485520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:05.497489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:05.499249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:05.600749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 636/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:56:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 08:56:05.607541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:05.705666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd177e0f-27fa-468c-b2d8-ab1066fc6d8b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:05.719768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:05.740586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:05.741145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:05.741425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:05.741697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:05.751866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:05.753310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:05.768533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:05.769667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:05.770785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:05.770785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:05.770785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:05.770785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 620/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:05.775481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:05.862371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:05.874397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acbc37-0d05-4e5a-af2e-119b1b568c25 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:05.908038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:05.908630 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:05.908907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:05.909187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:05.909800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:05.910165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:05.911555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:05.922463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 637/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:05.928234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:06.029013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91f3-c815-46ed-be02-c4a3ef08b293 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:06.062041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:06.062699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:06.062975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:06.063279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:06.069788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:06.071293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:56:06.082874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:06.084003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:06.085362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 621/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:06.090631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:06.189414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba697d-5080-4008-8220-28cfb982907a None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:06.222945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:06.223194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:06.223347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:06.223616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:06.224289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:06.224586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:06.225999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ec44f-0313-4782-afda-f7f81c5ecfdb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:06.238200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 638/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:06.244282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:06.271655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:06.346952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743bbe5-47aa-4f57-95d7-47020b2e3fc7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:06.381638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:06.382230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:06.382508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:06.382776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:06.383516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=97133f7b0f5b4320a41486752de09440) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:06.383824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:06.385244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294b3f6-153e-47c8-88e3-0723f7b6d670 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:06.391730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 622/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:56:06 2022] GET /identity/v3/role_assignments?user.id=97133f7b0f5b4320a41486752de09440&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:08.279985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432c28-45d0-4cd4-868f-873d57e91ed7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:08.280514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432c28-45d0-4cd4-868f-873d57e91ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:08.280514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432c28-45d0-4cd4-868f-873d57e91ed7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:08.281860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 639/1261] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:08.286956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c60599-221e-495c-9440-240d747b762a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:08.287379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c60599-221e-495c-9440-240d747b762a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:08.287549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c60599-221e-495c-9440-240d747b762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:08.305481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c0c60599-221e-495c-9440-240d747b762a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:08.306694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c0c60599-221e-495c-9440-240d747b762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:08.372941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 623/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:08.641754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432c28-45d0-4cd4-868f-873d57e91ed7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:08.653000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:08.741092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432c28-45d0-4cd4-868f-873d57e91ed7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:08.776152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:08.776762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:08.777155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:08.777429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:08.787449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:56:08.792184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:08.796546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:08.797605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 640/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:08.800035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c60599-221e-495c-9440-240d747b762a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:08.818928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:08.897557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c60599-221e-495c-9440-240d747b762a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:08.931447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:08.932046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:08.932321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:08.932589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:08.933329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:08.934272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:08.935791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:08.943863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 624/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:08.949855 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.016316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:09.050835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964084-0147-4083-9842-7d2565941ec4 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.085018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.085569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.085846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.086117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.097353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.098808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:09.114299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:09.115485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:09.116433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:09.116433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:09.116433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:09.116433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 641/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:09.122125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.201700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:09.220465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8813-8613-4119-8f9c-13616f380af8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.253262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.253858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.254135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.254402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.255030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:09.255362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.256723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.268388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 625/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:09.274229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.375550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce66d97-84b3-4d63-a9d7-1edb4872627d None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.394660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:09.408619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.409231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.409505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.409773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.417067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.419191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:56:09.430268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:09.431787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:09.432788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 642/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:09.437697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.537612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f42b2a-d349-49fb-8fd6-b3f5e6ac9ee3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.574651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.575286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.575679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.575951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.576673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:09.576981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.579486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.591211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:09.599739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 626/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:09.605594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.703596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7975e8-37f2-4d80-8569-09b66a42d0d4 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.736955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.737581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.737864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.738136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.748978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.750405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:09.765204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:09.771232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:09.772388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:09.772388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:09.772388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:09.772388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 643/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:09.772667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.871824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8102-a939-4f81-93ff-b1925c07d6b1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:09.906057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:09.906650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:09.906925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:09.907223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:09.907849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:09.908147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:09.909566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:09.920593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 627/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:09.926341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:09.958918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:10.025823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a70f3e-1c29-40f1-a43a-2ca4cd311562 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:10.061971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:10.062586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:10.062862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:10.063155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:10.069810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:10.071336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:56:10.083070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:10.084164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:10.085281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 644/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:56:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:10.090737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:10.150346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:10.191848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972c41-fdd0-4a39-9c0b-66b4fd302ab2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:10.224664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:10.225190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:10.225466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:10.225770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:10.226437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:10.226738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:10.228189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd21d95-20b2-46f0-8828-b3b1c701ca0e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:10.240355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 628/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:10.247068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:10.338284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:10.346883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361ae1-fbb5-4505-bde9-1b8c52476d6e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:10.381737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:10.382384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:10.382646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:10.382913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] PATH_INFO: `/v3/projects/c638c2670f324f9095be90304ed677a9/users/97133f7b0f5b4320a41486752de09440/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:10.384917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=c638c2670f324f9095be90304ed677a9, user_id=97133f7b0f5b4320a41486752de09440, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:10.386347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440a35-5cb9-49ef-bb24-269606126f96 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:10.476960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 645/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 08:56:10 2022] PUT /identity/v3/projects/c638c2670f324f9095be90304ed677a9/users/97133f7b0f5b4320a41486752de09440/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 08:56:12.387007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e06d-71f5-43db-b51c-de270d639a62 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:12.387567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e06d-71f5-43db-b51c-de270d639a62 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:12.387567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84e06d-71f5-43db-b51c-de270d639a62 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:12.388930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 629/1274] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:12.394088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae310-e924-493f-8211-211630cb762e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:12.399176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae310-e924-493f-8211-211630cb762e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:12.399176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafae310-e924-493f-8211-211630cb762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:12.413575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fafae310-e924-493f-8211-211630cb762e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:12.414758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fafae310-e924-493f-8211-211630cb762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:12.481286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 646/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:12.752118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84e06d-71f5-43db-b51c-de270d639a62 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:12.853519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84e06d-71f5-43db-b51c-de270d639a62 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:12.872214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:12.889958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:12.890582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:12.890858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:12.891154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:12.901276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 08:56:12.906655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:12.907874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:12.908937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 630/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 08:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:12.913880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae310-e924-493f-8211-211630cb762e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:12.993516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:13.013173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafae310-e924-493f-8211-211630cb762e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.046026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.046618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.046894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.047279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.047923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:13.048420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.050133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:13.058810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 647/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 08:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:13.064409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:13.164553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cf5d3-d609-4f3c-a609-5938d3164cbc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.198705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.200233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.200514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.200785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.210612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.212070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:13.226860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:13.228035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:13.228966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:13.228966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:13.228966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:13.228966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 631/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:13.234387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:13.333100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16271-efc1-4201-8761-1a47695de9a7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.365365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.365945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.366224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.366489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.367121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:13.367480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.368825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:13.381377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 648/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:13.387350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:13.418526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:13.487290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e85c4-110a-46ee-b08f-baaa84e07ae7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.521156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.521762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.522041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.522310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.529583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.531014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 08:56:13.541998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:13.543731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:13.544980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 632/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:13.551719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:13.555686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:13.654564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18154e3c-1d53-4448-97ce-de2105c8bccb None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.689288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.689879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.690180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.690446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.691176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:13.691474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.692912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:13.709720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 649/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 08:56:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:13.716402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:13.806606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:13.815658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdda11-d868-4738-b22c-a9fd5d501cc3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:13.849006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:13.849593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:13.849867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:13.850133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:13.915202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:13.916567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.004500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:14.015513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 633/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 08:56:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 08:56:14.021542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:14.085501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:14.121938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2330b1-1084-4a06-a179-c4712260b632 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:14.155236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:14.155861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:14.156172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:14.156406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:14.167237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:14.168838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 08:56:14.183459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:14.184645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 08:56:14.186062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 08:56:14.186062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 08:56:14.186062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:14.186062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 650/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 08:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:14.190500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:14.289627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97c9a-aa02-4c5c-8f73-57476fae8e71 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:14.323151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:14.323751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:14.324562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:14.324929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:14.325561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:14.325856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:14.327247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.338858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 634/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 08:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:14.345050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:14.444544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8b4bb-36eb-4d14-b35f-d75d833eaad9 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:14.458734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:14.478342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:14.478984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:14.479294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:14.479563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:14.486572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:14.493079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 08:56:14.500160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:14.501270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: ERROR keystone.server.flask.application Sep 01 08:56:14.502640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 651/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 08:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:14.507201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:14.557306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:14.607330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273be909-c1c0-490f-96bb-2777a2498f8a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:14.643483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:14.644105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:14.644414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:14.644690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:14.645400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:14.645716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:14.647231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979b2c-f7c7-4b46-9027-1e7ab7dd8965 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.666358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 635/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 08:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:14.667874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:14.780101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec44bc-abda-4558-bfc0-f33591588b9e None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:14.817095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:14.817820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:14.818167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:14.818505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:14.819365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] RBAC: Adding query filter params (role.id=8821f299abc1414b9817dedf7f7d0bea, scope.project.id=c638c2670f324f9095be90304ed677a9, user.id=97133f7b0f5b4320a41486752de09440) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:14.819673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:14.821141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad685d12-a8f3-4891-b295-654707d53189 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:14.828179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 652/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 08:56:14 2022] GET /identity/v3/role_assignments?user.id=97133f7b0f5b4320a41486752de09440&scope.project.id=c638c2670f324f9095be90304ed677a9&role.id=8821f299abc1414b9817dedf7f7d0bea => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:16.778731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924314-3e6e-4b0e-80c7-622e8a001974 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:16.779298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924314-3e6e-4b0e-80c7-622e8a001974 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:16.779298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85924314-3e6e-4b0e-80c7-622e8a001974 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:16.780649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 636/1288] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:16.785741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:16.786143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:16.786280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:16.805275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:16.805832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:16.834606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:16.872651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 653/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:17.139399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924314-3e6e-4b0e-80c7-622e8a001974 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:17.241848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85924314-3e6e-4b0e-80c7-622e8a001974 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:17.252979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:17.275692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:17.276304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:17.276582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:17.276854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:17.278246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:17.279658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:17.288749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:17.290155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:17.291403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 637/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:56:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:17.296332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:17.394246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93445b1-5c76-42e4-aeb0-865533eb5765 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:17.426326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:17.426912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:17.427212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:17.427482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:17.428090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:17.428384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:17.429737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254014b-a92c-4a98-8c6f-74073182c114 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:17.434940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 654/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 08:56:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:17.440845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:17.540855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9461b5-eefc-4b84-ae42-9daff692f547 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:17.574987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:17.575660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:17.575937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:17.576207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:17.576820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:17.577116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:17.578498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b86b-58d4-4cb1-84b7-ff6fb1f178bd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:17.582861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 638/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 08:56:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:19.459803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2382e9-34f8-4127-9996-dc327a679d0f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:19.460359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2382e9-34f8-4127-9996-dc327a679d0f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:19.460359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2382e9-34f8-4127-9996-dc327a679d0f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:19.461647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 655/1293] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:19.467729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42dfc-d61e-42e9-8918-49a0d672d5e8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:19.468147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42dfc-d61e-42e9-8918-49a0d672d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:19.468341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc42dfc-d61e-42e9-8918-49a0d672d5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:19.487388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3dc42dfc-d61e-42e9-8918-49a0d672d5e8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:19.488602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3dc42dfc-d61e-42e9-8918-49a0d672d5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:19.519167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:19.556097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 639/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:19.827968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2382e9-34f8-4127-9996-dc327a679d0f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:19.858879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:19.927277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2382e9-34f8-4127-9996-dc327a679d0f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:19.962053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:19.962663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:19.962939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:19.963234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:19.963888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:19.965325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e2364-e420-435a-ac3c-b90ada064cdc None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:20.101773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 656/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 08:56:19 2022] POST /identity/v3/services => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:56:21.995462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bb0c5-4065-4d47-9421-c76060a304ee None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:21.995981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bb0c5-4065-4d47-9421-c76060a304ee None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:21.995981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bb0c5-4065-4d47-9421-c76060a304ee None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:21.997641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 640/1296] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 08:56:22.002982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:22.003421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:22.003563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:22.021824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:22.023005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:22.095194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 657/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:22.364834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bb0c5-4065-4d47-9421-c76060a304ee None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:22.466062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bb0c5-4065-4d47-9421-c76060a304ee None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:22.498263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:22.499143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:22.499423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:22.499692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:22.500342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:22.501649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:22.510205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:22.511477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:22.513158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 641/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:56:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:22.518054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:22.536726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:22.620805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c6f59-eea3-4c13-9aee-5d1940e9b4e6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:22.658722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:22.659699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:22.660174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:22.660621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:22.661673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:22.662177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:22.664604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:22.671059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 658/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 08:56:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:22.677247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:22.709423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:22.777364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48db16-b5c1-41b9-bc67-54e418c7d827 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:22.810588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:22.811280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:22.811564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:22.811840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:22.812546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] RBAC: Adding query filter params (interface=public, service_id=e3c78e8d9d974b6f8d50c3a37fa4df96, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:22.812842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:22.814247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd78543-aa10-40bb-b772-51af1a2efc6d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:22.819195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 642/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 08:56:22 2022] GET /identity/v3/endpoints?service_id=e3c78e8d9d974b6f8d50c3a37fa4df96&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:22.825614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:22.910352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:22.928316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e19fa-216f-47d6-8617-a51f51859db7 None admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:22.961334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:22.961932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:22.962212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:22.962486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:22.963139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:22.964523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c88786-3ba6-4397-983c-3e46b085833e None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:22.971913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 659/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 08:56:22 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 08:56:24.880853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66f15d-5412-4c9e-b9ba-933eb5cfb1d5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:24.881370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66f15d-5412-4c9e-b9ba-933eb5cfb1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:24.881370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66f15d-5412-4c9e-b9ba-933eb5cfb1d5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:24.882683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 643/1302] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 08:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 08:56:24.889358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:24.893115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:24.893115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:24.908059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 08:56:24.909236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:56:24.975302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 660/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 08:56:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 08:56:25.241633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66f15d-5412-4c9e-b9ba-933eb5cfb1d5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:25.336894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:25.343635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66f15d-5412-4c9e-b9ba-933eb5cfb1d5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:25.377189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:25.383450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:25.383450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:25.383450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:25.383450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:25.383450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 08:56:25.389382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 08:56:25.390621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Sep 01 08:56:25.391990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 644/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 08:56:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 08:56:25.396549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:25.472617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:25.495233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73a72-edf3-430e-a88f-c6c585607422 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:25.529213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:25.529808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:25.530086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:25.530354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] PATH_INFO: `/v3/services` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:25.530967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 08:56:25.531398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:25.533642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef669c-148a-47da-8c71-482c0d376f76 None admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:25.541446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 661/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 08:56:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 08:56:25.547889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:56:25.646460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3470e-6dec-437b-b08a-7ac9577b0385 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:56:25.679767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:56:25.680355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:56:25.680632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:56:25.680900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:56:25.681552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:56:25.682951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180972ea-8b82-43f4-94c3-f9d104e39e8f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:56:25.752491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 645/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 08:56:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 08:56:35.536997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=30 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:56:35.748169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:09.598387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3656b-31cb-482d-999a-45d17a6910a1 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:09.598984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3656b-31cb-482d-999a-45d17a6910a1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:09.598984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3656b-31cb-482d-999a-45d17a6910a1 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:09.605421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3911-9018-4e62-b78a-f7a58a7fdd1a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:09.610588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3911-9018-4e62-b78a-f7a58a7fdd1a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:09.610588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3911-9018-4e62-b78a-f7a58a7fdd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:09.611202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 662/1307] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:09 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:09.683373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9a0d3911-9018-4e62-b78a-f7a58a7fdd1a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:09.684237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9a0d3911-9018-4e62-b78a-f7a58a7fdd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:09.792847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 646/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:02:10.091932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:10.092208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:10.092724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:10.094152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 663/1309] 10.222.0.46 () {54 vars in 965 bytes} [Thu Sep 1 09:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:10.100369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dc00-b33c-4759-b543-e49dd3193b01 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:10.100659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dc00-b33c-4759-b543-e49dd3193b01 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:10.100935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dc00-b33c-4759-b543-e49dd3193b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:10.204901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d4b7dc00-b33c-4759-b543-e49dd3193b01 None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:10.206082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d4b7dc00-b33c-4759-b543-e49dd3193b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:10.229172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:10.291635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 647/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Sep 1 09:02:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:10.301431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:10.303206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:10.430504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216eff-76e6-4748-920b-9c369154dd2b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:10.509199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:10.509716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:10.509992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:10.510259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:10.511881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:10.574871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:10.632987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:10.634435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980aa58-dc87-4ffe-bc04-9835722e378c service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:10.654309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 664/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:12.872611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:12.873145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:12.873145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:12.874682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 648/1312] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:12.881140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d061dbd-225b-4636-a2df-d453df4ff25c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:12.881690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d061dbd-225b-4636-a2df-d453df4ff25c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:12.881690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d061dbd-225b-4636-a2df-d453df4ff25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:12.970206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8d061dbd-225b-4636-a2df-d453df4ff25c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:12.972027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8d061dbd-225b-4636-a2df-d453df4ff25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:13.080480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 665/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:13.354323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:13.355871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:13.461965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a383b-360b-4673-966c-ee3fe56860ad None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:13.531327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:13.534212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:13.534808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:13.535086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:13.535400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:13.536980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:13.644779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:13.646224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514ee1-a313-4837-a1bc-605adc0b2c62 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:13.664301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 649/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:16.035905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:16.036424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:16.036424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:16.037762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 666/1315] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:16.042756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643cd9-b861-4df3-b239-3cdcf6efb065 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:16.043231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643cd9-b861-4df3-b239-3cdcf6efb065 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:16.043339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a643cd9-b861-4df3-b239-3cdcf6efb065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:16.107675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2a643cd9-b861-4df3-b239-3cdcf6efb065 None None] MFA Rules not processed for user `d67b28f7eb04449ba5b42d28ffe55408`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:16.108851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2a643cd9-b861-4df3-b239-3cdcf6efb065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:16.159898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:16.203473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 650/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:16.491397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:16.492935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:16.503132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:16.601103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8225f0-c36e-47fb-a8f5-c7b2bc21648d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:16.672464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:16.673644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:16.673992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:16.674261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:16.675885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:16.737431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:16.782913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:16.784365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe919c7-69f2-4fac-8230-8d3b560f0272 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:16.802887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 667/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:19.429663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:19.430200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:19.430200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:19.431517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 651/1318] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:19.436501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91b37-ccf3-4c06-8bbd-8a3ad41135a8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:19.436926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91b37-ccf3-4c06-8bbd-8a3ad41135a8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:19.437046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e91b37-ccf3-4c06-8bbd-8a3ad41135a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:19.504475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-96e91b37-ccf3-4c06-8bbd-8a3ad41135a8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:19.505669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-96e91b37-ccf3-4c06-8bbd-8a3ad41135a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:19.601030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 668/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:19.895696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:19.897663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:20.001577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263501f-d77b-41dd-a5bf-f2e26553704b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:20.072969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:20.073560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:20.073837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:20.074107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:20.075706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:20.082962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:20.184402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:20.185755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2a0d2-bb25-44c5-9aca-d0481356d483 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:20.204416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 652/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:23.038982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:23.039598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:23.039598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:23.040887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 669/1321] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:23.045971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2475d-53a6-456a-8b0e-1160c7a25c29 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:23.046426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2475d-53a6-456a-8b0e-1160c7a25c29 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:23.046523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2475d-53a6-456a-8b0e-1160c7a25c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:23.113308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3be2475d-53a6-456a-8b0e-1160c7a25c29 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:23.114491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3be2475d-53a6-456a-8b0e-1160c7a25c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:23.174159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:23.213512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 653/1322] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:23.493467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:23.495035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:23.515958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:23.607679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e5191-9d5e-496b-9c4a-590a7573b13f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:23.682501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:23.683163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:23.683403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:23.683670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:23.685248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:23.756354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:23.793415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:23.794853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb38d8-0792-4f55-9902-86a8fd285abf service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:23.813415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 670/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:25.827017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:25.827568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:25.827568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:25.828847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 654/1324] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:25.834670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4b0a-af33-417f-9595-b1e45b13de4f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:25.835093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4b0a-af33-417f-9595-b1e45b13de4f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:25.835373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4b0a-af33-417f-9595-b1e45b13de4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:25.905751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b68f4b0a-af33-417f-9595-b1e45b13de4f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:25.906947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b68f4b0a-af33-417f-9595-b1e45b13de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:26.004863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 671/1325] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:26.287716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:26.289225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:26.398014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402aa9-9466-4e99-a1ba-55007ea502d6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:26.481715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:26.482462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:26.482585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:26.483159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:26.484440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:26.502100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:26.594626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:26.596256 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948494e-a31a-408b-bc29-fb0490e6bb19 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:26.615462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 655/1326] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:28.579187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:28.580118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:28.580318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:28.581673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 672/1327] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:28.586949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172fe7-019e-4620-a3d0-83953fe5ee0d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:28.587452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172fe7-019e-4620-a3d0-83953fe5ee0d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:28.587544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172fe7-019e-4620-a3d0-83953fe5ee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:28.658527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1e172fe7-019e-4620-a3d0-83953fe5ee0d None None] MFA Rules not processed for user `d67b28f7eb04449ba5b42d28ffe55408`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:28.659790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1e172fe7-019e-4620-a3d0-83953fe5ee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:28.726676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:28.757505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 656/1328] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:29.040567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:29.042103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:29.091614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:29.172114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1973f-9581-4f45-8e84-e60d2aa77d2a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:29.254275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:29.254877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:29.255227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:29.255500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:29.257100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:29.340183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:29.371748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:29.373186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65788f-d075-4834-ae27-75785069277a service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:29.393531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 673/1329] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:31.912538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:31.913111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:31.913111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:31.914366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 657/1330] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:31.919643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48f4-2bfd-4f3a-a97f-73be8a80772c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:31.920074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48f4-2bfd-4f3a-a97f-73be8a80772c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:31.920175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e48f4-2bfd-4f3a-a97f-73be8a80772c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:31.987351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-227e48f4-2bfd-4f3a-a97f-73be8a80772c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:31.988552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-227e48f4-2bfd-4f3a-a97f-73be8a80772c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:32.086190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 674/1331] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:32.367425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:32.368991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:32.471695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ebeed-d544-4d09-af9d-d1718a157d79 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:32.543694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:32.544287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:32.544567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:32.544841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:32.546421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:32.573842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:32.655810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:32.657456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165af97-aef4-4374-861a-cba63d71e034 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:32.676820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 658/1332] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:34.648998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f182-5561-4f67-be40-56a98c363678 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:34.649559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f182-5561-4f67-be40-56a98c363678 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:34.649559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f182-5561-4f67-be40-56a98c363678 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:34.650888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 675/1333] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:34.656175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209f7ba-99cb-4c80-9ce6-e770acb855a5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:34.656597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209f7ba-99cb-4c80-9ce6-e770acb855a5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:34.656746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209f7ba-99cb-4c80-9ce6-e770acb855a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:34.723032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7209f7ba-99cb-4c80-9ce6-e770acb855a5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:34.724248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7209f7ba-99cb-4c80-9ce6-e770acb855a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:34.797655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:34.820699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 659/1334] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:35.101941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195f182-5561-4f67-be40-56a98c363678 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:35.103534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c195f182-5561-4f67-be40-56a98c363678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:35.144593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:35.209040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195f182-5561-4f67-be40-56a98c363678 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:35.283879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:35.284708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:35.284989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:35.285259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:35.286838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:35.369581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:35.394959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:35.396411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f186e7-cb74-4f62-bf6c-844c5f9c31c1 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:35.414888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 676/1335] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:37.940965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:37.941511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:37.941511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:37.942773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 660/1336] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:37.947883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86035b7f-5170-4853-bec1-e8b441229716 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:37.948278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86035b7f-5170-4853-bec1-e8b441229716 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:37.948438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86035b7f-5170-4853-bec1-e8b441229716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:38.014957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-86035b7f-5170-4853-bec1-e8b441229716 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:38.016428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-86035b7f-5170-4853-bec1-e8b441229716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:38.112587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 677/1337] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:38.396866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:38.398376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:38.499808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae8ab8-de1c-4528-a47b-36d24af8e27d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:38.571257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:38.571853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:38.572128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:38.572395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:38.573949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:38.609609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:38.681693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:38.683139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e95313-f8ce-48c5-9ecd-a572bef2756a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:38.701389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 661/1338] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:42.817471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:42.818030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:42.818030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:42.819516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 678/1339] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:42.824601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f3a20-004f-4e6b-8a96-760de634ed2e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:42.825040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f3a20-004f-4e6b-8a96-760de634ed2e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:42.825136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f3a20-004f-4e6b-8a96-760de634ed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:42.894947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-db1f3a20-004f-4e6b-8a96-760de634ed2e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:42.896166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-db1f3a20-004f-4e6b-8a96-760de634ed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:42.983344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:42.995592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 662/1340] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:02:43.281841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:43.283407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:43.333037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:43.390776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e355f-5951-4126-b549-4934a0b3c60a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:43.465803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:43.466418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:43.466763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:43.467119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:43.468722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:43.565969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:43.582252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:43.583718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d5683-8b4d-4df4-b2c1-5a2ef17c3044 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:43.602231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 679/1341] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:46.515282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:46.515805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:46.515805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:46.517329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 663/1342] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:46.522972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017ad4-0da8-4120-a72e-bc9834e253ec None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:46.523429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017ad4-0da8-4120-a72e-bc9834e253ec None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:46.523580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017ad4-0da8-4120-a72e-bc9834e253ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:46.592050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c3017ad4-0da8-4120-a72e-bc9834e253ec None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:46.593262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c3017ad4-0da8-4120-a72e-bc9834e253ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:46.692140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 680/1343] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:02:46.974689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:46.976247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:47.079124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13395879-9d1d-4524-96c4-65df4af4efb8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:47.158035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:47.158667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:47.159497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:47.159767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:47.161338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:47.205450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:47.275615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:47.277053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1683-d55c-4315-b8b0-6ea44129b00c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:47.295514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 664/1344] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:51.371644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:51.372195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:51.372195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:51.373652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 681/1345] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:51.379634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b88f-7a85-485c-a30a-62af9c9f9e95 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:51.380127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b88f-7a85-485c-a30a-62af9c9f9e95 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:51.380221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b88f-7a85-485c-a30a-62af9c9f9e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:51.455412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ebd0b88f-7a85-485c-a30a-62af9c9f9e95 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:51.456868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ebd0b88f-7a85-485c-a30a-62af9c9f9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:51.569061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:51.575484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 665/1346] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:02:51.860817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:02:51.863051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:51.922071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:51.968492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efd776-4e54-43cd-9694-a78cd113f6e9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:02:52.041850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:02:52.042445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:52.042718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:52.042985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:52.044615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:52.149388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:02:52.152775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:02:52.154268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf5a8e-5e61-46d6-ab47-8b6aa66fae80 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:02:52.172792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 682/1347] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:02:57.021347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b42de-df60-4a05-9d6f-de7af0f55e38 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:57.021927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b42de-df60-4a05-9d6f-de7af0f55e38 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:57.021927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b42de-df60-4a05-9d6f-de7af0f55e38 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:57.026101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 666/1348] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:02:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:02:57.030221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad00c7-6cd6-486a-8acb-4d6965cffd60 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:02:57.030677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad00c7-6cd6-486a-8acb-4d6965cffd60 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:02:57.030764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad00c7-6cd6-486a-8acb-4d6965cffd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:02:57.121322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-0aad00c7-6cd6-486a-8acb-4d6965cffd60 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:02:57.127150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0aad00c7-6cd6-486a-8acb-4d6965cffd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:02:57.321979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 683/1349] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:02:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:01.489048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76bfd-6a46-41b5-96bc-000dc70a53fc None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:01.490048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76bfd-6a46-41b5-96bc-000dc70a53fc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:01.490406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76bfd-6a46-41b5-96bc-000dc70a53fc None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:01.492012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 667/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:01.497455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c079a-c7cb-4607-bd80-86102204efb1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:01.497915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c079a-c7cb-4607-bd80-86102204efb1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:01.498298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c079a-c7cb-4607-bd80-86102204efb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:01.567364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:01.591668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5f4c079a-c7cb-4607-bd80-86102204efb1 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:01.592863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5f4c079a-c7cb-4607-bd80-86102204efb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:01.704921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 684/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:03:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:03.027569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2d48-9b0f-4f99-9fc1-447ce53e86e2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:03.028161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2d48-9b0f-4f99-9fc1-447ce53e86e2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:03.028161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2d48-9b0f-4f99-9fc1-447ce53e86e2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:03.029631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 668/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:03.035167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c45c-0270-4371-8ac0-62467a63a45d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:03.035648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c45c-0270-4371-8ac0-62467a63a45d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:03.035749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c45c-0270-4371-8ac0-62467a63a45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:03.115243 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fa67c45c-0270-4371-8ac0-62467a63a45d None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:03.116436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fa67c45c-0270-4371-8ac0-62467a63a45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:03.157928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:03.223628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 685/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:03:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:03:06.672169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033decb-f4a1-4da3-896a-a686f032e6b8 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:06.672732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033decb-f4a1-4da3-896a-a686f032e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:06.672732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033decb-f4a1-4da3-896a-a686f032e6b8 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:06.674136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 669/1354] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:06.680093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4120b6-c0d0-4fa0-bf15-9fa708e2b51a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:06.680528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4120b6-c0d0-4fa0-bf15-9fa708e2b51a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:06.680789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4120b6-c0d0-4fa0-bf15-9fa708e2b51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:06.764512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ec4120b6-c0d0-4fa0-bf15-9fa708e2b51a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:06.765867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ec4120b6-c0d0-4fa0-bf15-9fa708e2b51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:06.884810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 686/1355] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:07.193367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:07.194282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:07.194881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:07.197253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 670/1356] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 09:03:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:07.211877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603d2e7-0e16-4dff-9ce1-a6761ee347c8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:07.212827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603d2e7-0e16-4dff-9ce1-a6761ee347c8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:07.213339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603d2e7-0e16-4dff-9ce1-a6761ee347c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:07.230642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:07.297383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8603d2e7-0e16-4dff-9ce1-a6761ee347c8 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:07.298584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8603d2e7-0e16-4dff-9ce1-a6761ee347c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:07.420825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 687/1357] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 09:03:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:07.429478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:07.431041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:07.599206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693b68f-7e9e-4945-bfae-06bfa928ee6c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:07.679671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:07.722592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:07.723237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:07.723578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:07.723856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:07.725432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:07.818825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:07.820738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fd743-16f0-4302-9d84-6ff281654781 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:07.840077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 671/1358] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:09.981842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:09.982549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:09.982549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:09.983797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 688/1359] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:09.990432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ca0-feb3-465a-b9ac-88d647c72533 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:09.990959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ca0-feb3-465a-b9ac-88d647c72533 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:09.990959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ca0-feb3-465a-b9ac-88d647c72533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:10.060469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f9cc9ca0-feb3-465a-b9ac-88d647c72533 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:10.061815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f9cc9ca0-feb3-465a-b9ac-88d647c72533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:10.101961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:10.165149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 672/1360] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:10.464825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:10.466272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:10.543579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:10.640562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758dd4a9-8c5d-43ea-b4af-8cf6a7aadc57 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:10.766802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:10.767929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:10.768381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:10.769056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:10.771313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:10.797874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:10.884256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:10.886180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44baf3df-ad1b-4677-88a7-cdc7b719d0c9 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:10.910360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 689/1361] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:13.291572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57d472-2944-4fcf-b840-cd55227bf16e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:13.292142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57d472-2944-4fcf-b840-cd55227bf16e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:13.292142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57d472-2944-4fcf-b840-cd55227bf16e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:13.293445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 673/1362] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:13.298735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeafa63-33ca-447d-a4b1-ca9373ce312d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:13.299186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeafa63-33ca-447d-a4b1-ca9373ce312d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:13.299314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeafa63-33ca-447d-a4b1-ca9373ce312d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:13.369258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ffeafa63-33ca-447d-a4b1-ca9373ce312d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:13.370447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ffeafa63-33ca-447d-a4b1-ca9373ce312d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:13.476520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:13.476520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 690/1363] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:13.765833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:13.766407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:13.766783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:13.768819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 674/1364] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 09:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:13.774628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252957b1-7123-46f2-a8b1-cbe3fcaf7cda None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:13.774909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252957b1-7123-46f2-a8b1-cbe3fcaf7cda None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:13.775217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252957b1-7123-46f2-a8b1-cbe3fcaf7cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:13.859449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-252957b1-7123-46f2-a8b1-cbe3fcaf7cda None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:13.860644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-252957b1-7123-46f2-a8b1-cbe3fcaf7cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:13.984659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 691/1365] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 09:03:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:13.993298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:13.994970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:14.157166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248e4f-17a1-40d3-9305-7c802ef4188f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:14.179224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:14.277413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:14.278011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:14.278399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:14.278671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:14.280263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:14.377550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:14.384352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55184adf-b5dc-46cc-a92a-e123546ec5e9 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:14.405380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 675/1366] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:14.915583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16ec5f-27d1-44d6-961d-6297b040cf08 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:14.916151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16ec5f-27d1-44d6-961d-6297b040cf08 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:14.916151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16ec5f-27d1-44d6-961d-6297b040cf08 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:14.917473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 692/1367] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:14.922923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e4cb-f31c-4840-9f54-b9574f6d0691 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:14.923174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e4cb-f31c-4840-9f54-b9574f6d0691 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:14.923452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686e4cb-f31c-4840-9f54-b9574f6d0691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:14.957163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:14.996044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c686e4cb-f31c-4840-9f54-b9574f6d0691 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:14.997343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c686e4cb-f31c-4840-9f54-b9574f6d0691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:15.100605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 676/1368] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:15.685213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:15.685213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:15.685213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:15.687247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 693/1369] 10.222.0.46 () {54 vars in 964 bytes} [Thu Sep 1 09:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:15.691723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd220d-7263-46be-a00a-63cb1d1c759b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:15.692551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd220d-7263-46be-a00a-63cb1d1c759b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:15.692551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd220d-7263-46be-a00a-63cb1d1c759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:15.778213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b0cd220d-7263-46be-a00a-63cb1d1c759b None None] MFA Rules not processed for user `33f0625dda614821a5d9ca6c75de33d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:15.779492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b0cd220d-7263-46be-a00a-63cb1d1c759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:15.828412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:15.859025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 677/1370] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Sep 1 09:03:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:15.866858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:15.868407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:15.922506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:15.978869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb468e7-93cf-4488-a500-8571c7cb9b23 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:16.058056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:16.059013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:16.059477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:16.059927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:16.061680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:16.173273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:16.187328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:16.188796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0462d4-2e10-4d63-9c60-8431c5c08293 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:16.207980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 694/1371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:16.599245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:16.599538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:16.599822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:16.601195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 678/1372] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:03:16.606762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4980-9d65-4d17-908c-c8d352b65f83 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:16.607048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4980-9d65-4d17-908c-c8d352b65f83 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:16.607552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4980-9d65-4d17-908c-c8d352b65f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:16.674938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-760a4980-9d65-4d17-908c-c8d352b65f83 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:16.676807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-760a4980-9d65-4d17-908c-c8d352b65f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:16.774722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 695/1373] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:03:17.064264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:17.065818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:17.213654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e9f07-a6bf-4a80-a01c-1e7eea2facc3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:17.265544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:17.329274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:17.330005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:17.330289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:17.330689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:17.332420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:17.422963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:17.424596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f110a-9198-4731-bee3-ac9874f01905 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:17.443719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 679/1374] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:19.554382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:19.554823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:19.554916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:19.556449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 696/1375] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:19.562133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a568c2-8dca-495c-8da6-a35e4ce7a1d4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:19.562133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a568c2-8dca-495c-8da6-a35e4ce7a1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:19.562133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a568c2-8dca-495c-8da6-a35e4ce7a1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:19.627880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-47a568c2-8dca-495c-8da6-a35e4ce7a1d4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:19.629908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-47a568c2-8dca-495c-8da6-a35e4ce7a1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:19.636721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:19.737711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 680/1376] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:20.050286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:20.051990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:20.118044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:20.209579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a273-b281-4e1b-8e7d-5c4916b74043 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:20.333300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:20.334196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:20.334481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:20.334756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:20.336506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:20.357032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:20.435023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:20.436804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567a975-4eb5-454c-8415-e6c11d22098d service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:20.458070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 697/1377] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:22.521946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:22.522478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:22.522478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:22.523873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 681/1378] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:22.529185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72754e80-feda-4dcd-903f-0a75986097aa None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:22.529606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72754e80-feda-4dcd-903f-0a75986097aa None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:22.529717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72754e80-feda-4dcd-903f-0a75986097aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:22.605252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-72754e80-feda-4dcd-903f-0a75986097aa None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:22.606592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-72754e80-feda-4dcd-903f-0a75986097aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:22.717668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:22.725815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 698/1379] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:23.020787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:23.022273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:23.168121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:23.179824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf5cd7-1d64-4d9d-92f2-0b9ae1a59657 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:23.302622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:23.303257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:23.303535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:23.303806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:23.305642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:23.401841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:23.403590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942308ac-2d02-4985-a7bf-2e5740bc1619 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:23.415152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:23.428015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 682/1380] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:25.521810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:25.522387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:25.522488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:25.524247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 699/1381] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:25.529392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f28f5-d1b1-4eae-ab86-b3e244022e09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:25.529822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f28f5-d1b1-4eae-ab86-b3e244022e09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:25.529943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f28f5-d1b1-4eae-ab86-b3e244022e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:25.599200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-161f28f5-d1b1-4eae-ab86-b3e244022e09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:25.600392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-161f28f5-d1b1-4eae-ab86-b3e244022e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:25.714081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 683/1382] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:26.049999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:26.051643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:26.202193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee781ca-b87f-4bb6-924e-b33f0bd6f748 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:26.278045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:26.326666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:26.327496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:26.328141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:26.328840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:26.330924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:26.442429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:26.444338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f423ca-f23c-4a20-9d52-6a9c264d4521 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:26.465992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 700/1383] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:28.397028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:28.397706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:28.397706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:28.398943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 684/1384] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:28.404351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63561f59-2500-4bf9-b358-fcfa82bb4160 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:28.404841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63561f59-2500-4bf9-b358-fcfa82bb4160 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:28.405028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63561f59-2500-4bf9-b358-fcfa82bb4160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:28.485183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-63561f59-2500-4bf9-b358-fcfa82bb4160 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:28.486647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-63561f59-2500-4bf9-b358-fcfa82bb4160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:28.525823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:28.607535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 701/1385] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:28.956703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:28.959056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:29.055263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:29.131997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b62c-f563-4162-b0d5-f545945b5721 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:29.257429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:29.258030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:29.258304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:29.258570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:29.260188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:29.297847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:29.351523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:29.353124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011f318-d590-4240-8e5e-e36cf015c805 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:29.372203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 685/1386] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:31.424541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcda8f0-9fd6-4d06-82c4-824fa0d72592 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.425103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcda8f0-9fd6-4d06-82c4-824fa0d72592 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.425103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcda8f0-9fd6-4d06-82c4-824fa0d72592 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.426486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 702/1387] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:31.431724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.432137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.432287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.488667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653f3e9-2c37-4cf9-83a2-0cd224bea704 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.488875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653f3e9-2c37-4cf9-83a2-0cd224bea704 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.489147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653f3e9-2c37-4cf9-83a2-0cd224bea704 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.490491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 703/1388] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:03:31.495321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991bfe3-b244-4dd9-8faf-a621358eae2e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.495626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991bfe3-b244-4dd9-8faf-a621358eae2e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.495893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991bfe3-b244-4dd9-8faf-a621358eae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.500410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:31.501617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:31.565737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b991bfe3-b244-4dd9-8faf-a621358eae2e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:31.567596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b991bfe3-b244-4dd9-8faf-a621358eae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:31.604322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 686/1389] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:31.670066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 704/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:03:31.894748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:31.896043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:31.910179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:31.970254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cb677-4034-4071-b5f2-019e1b1622d7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.970546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cb677-4034-4071-b5f2-019e1b1622d7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.970827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cb677-4034-4071-b5f2-019e1b1622d7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.972214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 705/1391] 10.222.0.46 () {54 vars in 967 bytes} [Thu Sep 1 09:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:03:31.977144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:31.977429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:31.977703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:31.993560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:32.017201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5332a-3f85-41eb-a254-b111c10dd9b7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:32.056821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] MFA Rules not processed for user `c1c10f44607a4d8080891c89b9b88345`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:32.058069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:32.132970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:32.133484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:32.133764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:32.134036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:32.135660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:32.139070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 706/1392] 10.222.0.46 () {58 vars in 1064 bytes} [Thu Sep 1 09:03:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:03:32.144547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:32.147841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:32.155327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8ebb6-831e-4c25-8b4c-177b591ccdcb None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:32.229514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:32.231141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0839-3174-4282-836e-29de23da1d93 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:32.231444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:32.232047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:32.232321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:32.232588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:32.234163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:32.249312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 687/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:32.265414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:32.348566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:32.350218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66e7f1-ea9d-4957-8394-4a63cc19fd9e service glance] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:32.369625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 707/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:03:33.383810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e48b-418a-4e3d-8781-7357b9c5f86b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:33.384364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e48b-418a-4e3d-8781-7357b9c5f86b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:33.384364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305e48b-418a-4e3d-8781-7357b9c5f86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:33.452352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1305e48b-418a-4e3d-8781-7357b9c5f86b None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:33.453542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1305e48b-418a-4e3d-8781-7357b9c5f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:33.543118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:33.546286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 688/1395] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:03:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:33.595289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:33.595802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:33.595909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:33.597175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 708/1396] 10.222.0.46 () {54 vars in 943 bytes} [Thu Sep 1 09:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:33.605175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f545-ab2a-4ada-ad45-d6b60726c9d9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:33.606796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f545-ab2a-4ada-ad45-d6b60726c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:33.606796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f545-ab2a-4ada-ad45-d6b60726c9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:33.685111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5887f545-ab2a-4ada-ad45-d6b60726c9d9 None None] MFA Rules not processed for user `0c95310969054beab60e99fd71a34aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:33.686277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5887f545-ab2a-4ada-ad45-d6b60726c9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:33.793956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 689/1397] 10.222.0.46 () {58 vars in 1040 bytes} [Thu Sep 1 09:03:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:33.804022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:33.805660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:33.942076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:33.944935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456534a2-d340-4d14-97a5-fa31316a847f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:34.052525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:34.053129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:34.053397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:34.053659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:34.055474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:34.141363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:34.142980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd404503-b094-487a-8286-1bb031f284ae service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:34.151487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 709/1398] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:03:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:03:34.338633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:34.338914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:34.339216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:34.340576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 690/1399] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:34.345975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e47c-e46d-4703-b05b-5150d179e160 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:34.346286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e47c-e46d-4703-b05b-5150d179e160 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:34.346564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e47c-e46d-4703-b05b-5150d179e160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:34.384134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:34.416813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3769e47c-e46d-4703-b05b-5150d179e160 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:34.418004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3769e47c-e46d-4703-b05b-5150d179e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:34.521865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 710/1400] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:34.814823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:34.816403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:34.885763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:34.989478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490f237-f374-4de4-a7fb-b062e957b6a6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:35.119620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:35.120336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:35.120618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:35.120892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:35.122531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:35.142139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:35.226156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:35.229142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770551b-a284-42ff-9a22-ec48aa8f262b service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:35.248983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 691/1401] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:37.433286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efab28-0236-4346-a41a-707ad540621f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:37.433906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efab28-0236-4346-a41a-707ad540621f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:37.433906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efab28-0236-4346-a41a-707ad540621f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:37.435199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 711/1402] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:37.440558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aefc9-f374-4f2d-9e2d-8a6830fdf076 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:37.441567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aefc9-f374-4f2d-9e2d-8a6830fdf076 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:37.441772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aefc9-f374-4f2d-9e2d-8a6830fdf076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:37.520404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a79aefc9-f374-4f2d-9e2d-8a6830fdf076 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:37.521606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a79aefc9-f374-4f2d-9e2d-8a6830fdf076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:37.624681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:37.639146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 692/1403] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:37.929160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efab28-0236-4346-a41a-707ad540621f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:37.930923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-66efab28-0236-4346-a41a-707ad540621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:38.058335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:38.105627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efab28-0236-4346-a41a-707ad540621f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:38.234161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:38.234161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:38.234161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:38.234554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:38.236097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:38.315928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:38.336955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:38.338579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f8692-dea7-4e1d-8a76-6d14d3e510a1 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:38.357092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 712/1404] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:40.489941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:40.490477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:40.490477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:40.493100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 693/1405] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:03:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:40.496970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530195-b09f-4446-b14e-b44acc7ff11c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:40.497424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530195-b09f-4446-b14e-b44acc7ff11c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:40.497510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1530195-b09f-4446-b14e-b44acc7ff11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:40.571667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f1530195-b09f-4446-b14e-b44acc7ff11c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:40.572910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f1530195-b09f-4446-b14e-b44acc7ff11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:40.688171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 713/1406] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:40.985252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:03:40.986771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:41.179830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da664e8-285f-4174-b965-d8afa2095286 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:03:41.273953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:41.324998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:03:41.325920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:41.326502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:41.327088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:41.329458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:41.490333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:03:41.491960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a4dc2-30dc-457e-997a-be2f31bbf7a1 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:03:41.523418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 694/1407] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:03:50.686320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=79 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:51.509333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=64 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:57.933477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b78eff-24cf-4bd9-9b2d-9cc457e4070a None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:57.934008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b78eff-24cf-4bd9-9b2d-9cc457e4070a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:57.934008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b78eff-24cf-4bd9-9b2d-9cc457e4070a None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:57.935335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 714/1408] 10.222.0.46 () {54 vars in 918 bytes} [Thu Sep 1 09:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:57.940399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a6a1-2f2e-46cf-a67d-715bbd6a1d96 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:57.940842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a6a1-2f2e-46cf-a67d-715bbd6a1d96 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:57.940949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a6a1-2f2e-46cf-a67d-715bbd6a1d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:58.016661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6d18a6a1-2f2e-46cf-a67d-715bbd6a1d96 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:58.017891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6d18a6a1-2f2e-46cf-a67d-715bbd6a1d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:58.122984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 695/1409] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:03:58.137215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7003a4-5858-4605-b531-01de7e12806d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:58.137500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7003a4-5858-4605-b531-01de7e12806d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:58.137772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7003a4-5858-4605-b531-01de7e12806d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:58.139114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 715/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu Sep 1 09:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:03:58.143606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeb5ac-65d8-4b5a-9d31-fcf20a25d4b1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:03:58.143908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeb5ac-65d8-4b5a-9d31-fcf20a25d4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:03:58.144181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbeb5ac-65d8-4b5a-9d31-fcf20a25d4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:03:58.215767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:03:58.218590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3dbeb5ac-65d8-4b5a-9d31-fcf20a25d4b1 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:03:58.219813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3dbeb5ac-65d8-4b5a-9d31-fcf20a25d4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:03:58.333776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 696/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:00.705557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:00.706122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:00.706122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:00.707499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 716/1412] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:00.713398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368b62-0382-4ea3-902a-554f3fa9ee62 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:00.713812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368b62-0382-4ea3-902a-554f3fa9ee62 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:00.713936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368b62-0382-4ea3-902a-554f3fa9ee62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:00.778716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-40368b62-0382-4ea3-902a-554f3fa9ee62 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:00.780007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-40368b62-0382-4ea3-902a-554f3fa9ee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:00.835683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:00.877143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 697/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:01.180262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:01.181772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:01.330124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f3fa-3498-4de2-a8de-24ec00372023 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:01.404523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:01.445878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:01.446407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:01.446682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:01.446950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:01.448607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:01.540156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:01.541760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fd19f-04eb-4b55-bac9-6fe38e12da9f service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:01.560241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 717/1414] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:02.467360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:02.467940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:02.467940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:02.469329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 698/1415] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:04:02.474960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff838-250c-4fba-802d-d6d0a404d5d4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:02.475427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff838-250c-4fba-802d-d6d0a404d5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:02.475542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cff838-250c-4fba-802d-d6d0a404d5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:02.550142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-38cff838-250c-4fba-802d-d6d0a404d5d4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:02.551371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-38cff838-250c-4fba-802d-d6d0a404d5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:02.579706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:02.649801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 718/1416] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:02.804430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:02.805985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:02.952018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a5285-374f-49cb-be97-f1677899a7fa None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:02.992649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:03.064733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:03.065279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:03.065551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:03.065818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:03.067508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:03.154027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:03.155641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46fc21-63d2-41a6-91ec-8785853aa7d0 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:03.173395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 699/1417] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:04.047350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:04.047909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:04.047909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:04.049285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 719/1418] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:04.054932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb628-c8cc-4c0f-8258-786d6f0650f0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:04.055472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb628-c8cc-4c0f-8258-786d6f0650f0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:04.055572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb628-c8cc-4c0f-8258-786d6f0650f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:04.108333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:04.129785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-61fcb628-c8cc-4c0f-8258-786d6f0650f0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:04.132187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-61fcb628-c8cc-4c0f-8258-786d6f0650f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:04.225829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 700/1419] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:04.376622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:04.378046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:04.527735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b3822-8d3f-4a0a-8539-8015d884e5df None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:04.540802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:04.643055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:04.643615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:04.643887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:04.644152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:04.646698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:04.737475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:04.739055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b3545-63bb-41c9-9054-5a2751c55737 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:04.757911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 720/1420] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:05.607348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:05.616226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:05.616226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:05.616226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 701/1421] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:05.616530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a57f-8515-4625-af2a-b9fd7660fa08 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:05.616530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a57f-8515-4625-af2a-b9fd7660fa08 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:05.616530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954a57f-8515-4625-af2a-b9fd7660fa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:05.635058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:05.693222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7954a57f-8515-4625-af2a-b9fd7660fa08 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:05.694436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7954a57f-8515-4625-af2a-b9fd7660fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:05.791945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 721/1422] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:05.941152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:05.943144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:06.053611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:06.089877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2801e3b-4b28-4f18-86ce-1e8e7e3efefd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:06.200059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:06.200585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:06.200863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:06.201133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:06.202705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:06.277755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:06.290492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:06.292137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1ff26-035d-49a8-93a7-d7ae0a246c3d service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:06.311516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 702/1423] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:07.175612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:07.176267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:07.176267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:07.177607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 722/1424] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:07.182554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397a98-4d10-47c2-a57a-f18f3687cc2a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:07.182992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397a98-4d10-47c2-a57a-f18f3687cc2a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:07.183132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397a98-4d10-47c2-a57a-f18f3687cc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:07.258061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5c397a98-4d10-47c2-a57a-f18f3687cc2a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:07.259298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5c397a98-4d10-47c2-a57a-f18f3687cc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:07.356301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 703/1425] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:07.514000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:07.515486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:07.742441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:07.818833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6da7fa-7e85-4b91-8372-f0ca3a261f0e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:07.943096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:07.943723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:07.943993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:07.944382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:07.945983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:08.001774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:08.050404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:08.052727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b45922-0e3d-4dfb-b4b0-4bc8f858f587 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:08.072302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 723/1426] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:08.961716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:08.962289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:08.962398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:08.963709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 704/1427] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:08.968670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6958-0ae1-44aa-8f1c-92293263c913 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:08.969169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6958-0ae1-44aa-8f1c-92293263c913 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:08.969272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db6958-0ae1-44aa-8f1c-92293263c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:09.046697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-27db6958-0ae1-44aa-8f1c-92293263c913 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:09.047937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-27db6958-0ae1-44aa-8f1c-92293263c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:09.144707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 724/1428] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:04:09.296241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:09.297757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:09.347603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:09.445266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18fd19-a5e3-47c0-8a16-218cb18ffa0e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:09.558370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:09.558978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:09.559287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:09.559556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:09.561138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:09.568424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:09.651831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:09.653450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767f994-37d8-4fea-8cf3-348606f8186e service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:09.671748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 705/1429] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:10.545883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:10.546476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:10.546476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:10.547790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 725/1430] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:10.553747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae43c3c-2c09-49a1-b54e-7e6f7db4db41 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:10.557321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae43c3c-2c09-49a1-b54e-7e6f7db4db41 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:10.557321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae43c3c-2c09-49a1-b54e-7e6f7db4db41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:10.631444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-eae43c3c-2c09-49a1-b54e-7e6f7db4db41 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:10.632695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eae43c3c-2c09-49a1-b54e-7e6f7db4db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:10.701834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:10.734213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 706/1431] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:10.885462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:10.886947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:10.905624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:11.029668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b86eda-c625-4f9f-affd-8a6e682dce5c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:11.114384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:11.139549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:11.140205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:11.140670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:11.140983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:11.143015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:11.231385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:11.233050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d81bf-3594-44de-b00e-af252ad7fd78 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:11.250712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 726/1432] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:12.132713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:12.133309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:12.133309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:12.134622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 707/1433] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:12.139463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b535a-1f15-4b70-a132-4c13b582e592 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:12.139897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b535a-1f15-4b70-a132-4c13b582e592 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:12.140009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b535a-1f15-4b70-a132-4c13b582e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:12.215956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6f9b535a-1f15-4b70-a132-4c13b582e592 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:12.217156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6f9b535a-1f15-4b70-a132-4c13b582e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:12.258747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:12.312923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 727/1434] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:12.462672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:12.464128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:12.606605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8060a-5cb5-40af-b1f8-970810b4f51c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:12.661127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:12.719347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:12.719875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:12.720152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:12.720419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:12.721983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:12.810868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:12.812547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b97fae-c06c-4612-9a7d-0c16acb06dd2 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:12.831515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 708/1435] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:13.700337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:13.700917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:13.700917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:13.702195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 728/1436] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:13.707132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523bf3-ee71-481e-8c6c-198998e2191e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:13.707568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523bf3-ee71-481e-8c6c-198998e2191e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:13.707692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523bf3-ee71-481e-8c6c-198998e2191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:13.783625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fb523bf3-ee71-481e-8c6c-198998e2191e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:13.784815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fb523bf3-ee71-481e-8c6c-198998e2191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:13.786923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:13.882729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 709/1437] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:14.037659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:14.039248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:14.182239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb22b-08ff-4b12-a0d2-679a9306bde2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:14.208184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:14.291776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:14.292374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:14.292647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:14.292910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:14.294471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:14.380974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:14.382535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf45a64-660b-4784-a879-2a5610bd1b14 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:14.400128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 729/1438] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:16.240182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:16.240746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:16.240746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:16.242074 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 710/1439] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:16.247279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f88f1-b2cd-4ad0-9ed8-7d655a827c1a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:16.247683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f88f1-b2cd-4ad0-9ed8-7d655a827c1a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:16.247832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f88f1-b2cd-4ad0-9ed8-7d655a827c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:16.289547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:16.313853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-eb7f88f1-b2cd-4ad0-9ed8-7d655a827c1a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:16.315040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-eb7f88f1-b2cd-4ad0-9ed8-7d655a827c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:16.410848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 730/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:16.704322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:16.705854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:16.813135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8d305-ac72-4f6f-85e4-ff079d484a52 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:16.852244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:16.895976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:16.896581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:16.896858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:16.897129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:16.899065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:17.007871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:17.009448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d91f6a-b9ab-4da0-bcdd-137a65a02f23 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:17.027536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 711/1441] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:04:19.098776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:19.099433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:19.099433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:19.101225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 731/1442] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:19.106539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b53b1-2411-4ecc-b2b2-776e28a5f29b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:19.106983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b53b1-2411-4ecc-b2b2-776e28a5f29b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:19.107093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b53b1-2411-4ecc-b2b2-776e28a5f29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:19.171697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-073b53b1-2411-4ecc-b2b2-776e28a5f29b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:19.172880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-073b53b1-2411-4ecc-b2b2-776e28a5f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:19.185867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:19.268460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 712/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:19.552273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:19.553937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:19.658200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945450f-2996-4b7b-b7fd-f5cfa84fa097 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:19.671025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:19.732572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:19.733215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:19.733509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:19.733756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:19.735352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:19.844153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:19.845492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed11ec-d2f5-46c2-8d85-a03d28585718 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:19.863269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 732/1444] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:22.119199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:22.119759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:22.119759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:22.121113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 713/1445] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:22.126333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159740a8-dcd9-4f36-9a9a-aa5fef346bad None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:22.126770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159740a8-dcd9-4f36-9a9a-aa5fef346bad None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:22.126892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159740a8-dcd9-4f36-9a9a-aa5fef346bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:22.174442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:22.192679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-159740a8-dcd9-4f36-9a9a-aa5fef346bad None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:22.193898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-159740a8-dcd9-4f36-9a9a-aa5fef346bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:22.289003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 733/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:22.573896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:22.575462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:22.680932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c10d45-9b17-4cd0-b94d-6d8ca04b1ef5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:22.720287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:22.756845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:22.757432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:22.757710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:22.757980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:22.759609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:22.868555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:22.869931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbd707-a762-495e-8099-3b7c5c0ab159 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:22.888754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 714/1447] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:24.905609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:24.906135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:24.906135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:24.907470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 734/1448] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:04:24.912899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149129-ccf4-4f37-b14b-70f6f90fc749 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:24.914005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149129-ccf4-4f37-b14b-70f6f90fc749 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:24.914454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149129-ccf4-4f37-b14b-70f6f90fc749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:24.982322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1d149129-ccf4-4f37-b14b-70f6f90fc749 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:24.983555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1d149129-ccf4-4f37-b14b-70f6f90fc749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:25.004185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:25.084876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 715/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:25.365307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:25.367149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:25.471449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbcd4-cddf-488d-a4c7-4f33e7276785 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:25.491198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:25.546670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:25.547299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:25.547575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:25.547843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:25.549422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:25.658629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:25.660087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9128cf1-777a-453e-a0a6-4d03c0bc47e4 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:25.678089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 735/1450] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:27.742650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:27.743228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:27.743228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:27.744595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 716/1451] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:27.749882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3756a6-fa12-4dfe-918c-c0c91e09f72c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:27.750318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3756a6-fa12-4dfe-918c-c0c91e09f72c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:27.750424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3756a6-fa12-4dfe-918c-c0c91e09f72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:27.813544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:27.815996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-0a3756a6-fa12-4dfe-918c-c0c91e09f72c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:27.817196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0a3756a6-fa12-4dfe-918c-c0c91e09f72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:27.913016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 736/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:28.194209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:28.196953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:28.298135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f89b14-9455-4da3-893d-6f3c8108d182 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:28.346797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:28.372154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:28.372745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:28.376123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:28.376123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:28.376123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:28.482827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:28.484241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab04290-61a2-4a67-a860-c071cabc25bd service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:28.502093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 717/1453] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:30.459868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:30.461072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:30.461072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:30.461856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 737/1454] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:30.467141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15189a-3978-4fbc-ad48-3b1b6d5b6f85 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:30.467579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15189a-3978-4fbc-ad48-3b1b6d5b6f85 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:30.467726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15189a-3978-4fbc-ad48-3b1b6d5b6f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:30.534545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ed15189a-3978-4fbc-ad48-3b1b6d5b6f85 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:30.535772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ed15189a-3978-4fbc-ad48-3b1b6d5b6f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:30.566796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:30.635052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 718/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:30.928495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:30.930542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:31.032955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be188399-64b8-4379-b7d2-235f7e5ec6cc None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:31.061368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:31.107023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:31.107651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:31.107926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:31.108197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:31.109777 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:31.216866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:31.218208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c97e63-da24-4f59-b66a-5c514bbb79fd service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:31.236742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 738/1456] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:33.201444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:33.201974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:33.201974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:33.203297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 719/1457] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:04:33.208541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069393-415c-43b2-9988-25f9a00b2af3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:33.208964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069393-415c-43b2-9988-25f9a00b2af3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:33.209098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069393-415c-43b2-9988-25f9a00b2af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:33.274697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-11069393-415c-43b2-9988-25f9a00b2af3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:33.275909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-11069393-415c-43b2-9988-25f9a00b2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:33.283991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:33.371841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 739/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:33.651357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:33.655169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:33.758539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d36199-b649-4b86-8e96-0f495555ac36 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:33.813699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:33.831593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:33.832194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:33.832525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:33.832826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:33.834404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:33.941804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:33.943276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696946-1994-477a-a43d-f73825afe278 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:33.961683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 720/1459] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:35.906474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0db-f74c-41b3-b6cd-b9a51907a800 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:35.907003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0db-f74c-41b3-b6cd-b9a51907a800 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:35.907003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0db-f74c-41b3-b6cd-b9a51907a800 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:35.908355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 740/1460] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:35.913405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac468a-108e-4230-a39a-f3ccfb23f238 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:35.914227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac468a-108e-4230-a39a-f3ccfb23f238 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:35.914227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac468a-108e-4230-a39a-f3ccfb23f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:35.978970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c2ac468a-108e-4230-a39a-f3ccfb23f238 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:35.980194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c2ac468a-108e-4230-a39a-f3ccfb23f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:36.017929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:36.077504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 721/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:36.380682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:36.380961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:36.381235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:36.383196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 741/1462] 10.222.0.46 () {54 vars in 965 bytes} [Thu Sep 1 09:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:36.389372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2f9a-e4cf-434e-a2b6-ff93d6a33af5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:36.389657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2f9a-e4cf-434e-a2b6-ff93d6a33af5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:36.389932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae2f9a-e4cf-434e-a2b6-ff93d6a33af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:36.469369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-29ae2f9a-e4cf-434e-a2b6-ff93d6a33af5 None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:36.470562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-29ae2f9a-e4cf-434e-a2b6-ff93d6a33af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:36.545193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 722/1463] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Sep 1 09:04:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:36.554463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:36.556186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:36.660257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e23a1a-e626-49ce-9b2f-195df96ecde6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:36.699996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:36.736475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:36.737103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:36.737374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:36.737641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:36.739237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:36.849776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:36.851359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8606-6913-4ac3-bac3-6cacd4455a81 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:36.870331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 742/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:39.196663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:39.197209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:39.197209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:39.198482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:39.203749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce43f-21a6-42eb-bc4b-b06323d9b73e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:39.204138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce43f-21a6-42eb-bc4b-b06323d9b73e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:39.204293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce43f-21a6-42eb-bc4b-b06323d9b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:39.269975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bdbce43f-21a6-42eb-bc4b-b06323d9b73e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:39.271460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bdbce43f-21a6-42eb-bc4b-b06323d9b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:39.290668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:39.369412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:39.664653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:39.666253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:39.703471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:39.771470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d07b6a-8325-4b78-967f-6c1a6867ba4c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:39.846090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:39.846600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:39.846883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:39.847198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:39.848807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:39.932286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:39.959424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:39.961579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301674b-b390-40b6-9ac9-0d0f132dcfb7 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:39.979767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 724/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:41.925521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:41.926095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:41.926095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:41.927469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:04:41.933539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-ab31-49ce-9cc9-bfb69517fffc None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:41.933929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-ab31-49ce-9cc9-bfb69517fffc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:41.934126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-ab31-49ce-9cc9-bfb69517fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:42.000168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8e616a1b-ab31-49ce-9cc9-bfb69517fffc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:04:42.001383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8e616a1b-ab31-49ce-9cc9-bfb69517fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:42.096368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:04:42.381956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:04:42.384010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:42.485319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795213e4-4ac1-42c8-8013-6757607d293f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:04:42.531467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:42.558226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:04:42.558869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:04:42.559247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:04:42.559516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:04:42.561093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:04:42.669893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:04:42.671276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8229c2e-ca5d-4e2b-b027-786c18fd871a service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:04:42.691440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 745/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:04:52.094573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:04:52.689172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=66 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:02.270907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7613c-b821-491a-8a88-ce7b65b00de5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:02.271512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7613c-b821-491a-8a88-ce7b65b00de5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:02.271512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7613c-b821-491a-8a88-ce7b65b00de5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:02.273128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 726/1471] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:02.279241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714beb5-3fd9-49a2-8f0a-b7e5e9794e03 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:02.279655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714beb5-3fd9-49a2-8f0a-b7e5e9794e03 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:02.279775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714beb5-3fd9-49a2-8f0a-b7e5e9794e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:02.344630 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-2714beb5-3fd9-49a2-8f0a-b7e5e9794e03 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:02.345809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2714beb5-3fd9-49a2-8f0a-b7e5e9794e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:02.442360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 746/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:02.739539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:02.739856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:02.740131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:02.741468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 727/1473] 10.222.0.46 () {54 vars in 967 bytes} [Thu Sep 1 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:02.746941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d35-604f-4a99-a7ae-5458b515f8a1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:02.747261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d35-604f-4a99-a7ae-5458b515f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:02.747528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f900d35-604f-4a99-a7ae-5458b515f8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:02.823620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8f900d35-604f-4a99-a7ae-5458b515f8a1 None None] MFA Rules not processed for user `c1c10f44607a4d8080891c89b9b88345`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:02.824741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8f900d35-604f-4a99-a7ae-5458b515f8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:02.844623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:02.900663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 747/1474] 10.222.0.46 () {58 vars in 1064 bytes} [Thu Sep 1 09:05:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:02.908287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:02.909868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:03.012369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3041e40-3bcd-4aaf-b003-06a26b8834fc None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:03.087397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:03.087928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:03.088207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:03.088474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:03.090083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:03.146790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:03.201750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:03.203222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fd566-0c91-475b-a3cf-cd1903c63681 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:03.221003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 728/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:05.211885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:05.212852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:05.212852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:05.213887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 748/1476] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:05.219007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249459f-19ea-4cac-b74d-2d9c4aa99cca None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:05.219482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249459f-19ea-4cac-b74d-2d9c4aa99cca None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:05.219600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249459f-19ea-4cac-b74d-2d9c4aa99cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:05.286630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2249459f-19ea-4cac-b74d-2d9c4aa99cca None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:05.287842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2249459f-19ea-4cac-b74d-2d9c4aa99cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:05.383830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 729/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:05.658235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:05.659818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:05.786242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff5a81-c883-4efb-8866-abeebd00e724 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:05.835651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:05.880270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:05.880867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:05.881138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:05.881403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:05.882055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:05.884482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:05:05.884482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:05:05.884482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:05:05.885645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fe3d3-9f97-46d7-a2ba-3cc632be0732 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:05.905123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 749/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 09:05:05 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:05:07.812207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:07.812721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:07.812721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:07.814039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 730/1479] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:07.819285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50cd7-8362-45ac-8967-5e925f4e1dd9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:07.819718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50cd7-8362-45ac-8967-5e925f4e1dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:07.819824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf50cd7-8362-45ac-8967-5e925f4e1dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:07.884601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ecf50cd7-8362-45ac-8967-5e925f4e1dd9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:07.885794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ecf50cd7-8362-45ac-8967-5e925f4e1dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:07.980430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 750/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:08.270608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:08.272161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:08.282998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:08.421448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b260fb-997c-4043-a7bf-e3b5dad843f1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:08.502227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:08.534649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:08.535272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:08.535549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:08.535817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:08.537396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:08.630522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:08.632213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52e60-20b2-4a27-83d2-1106cf4e643a service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:08.650631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 731/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:10.952350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:10.952847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:10.952847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:10.954154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 751/1482] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:10.959274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd572c9-ed91-42f3-b85e-970c02f2adb5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:10.959705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd572c9-ed91-42f3-b85e-970c02f2adb5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:10.959817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd572c9-ed91-42f3-b85e-970c02f2adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:11.024865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ddd572c9-ed91-42f3-b85e-970c02f2adb5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:11.026059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ddd572c9-ed91-42f3-b85e-970c02f2adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:11.071520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:11.121692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 732/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:11.410447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:11.412039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:11.436732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:11.517168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2878-30d7-49b2-8f4d-acbe38975ed1 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:11.593070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:11.593665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:11.593938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:11.594202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:11.595802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:11.672060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:11.706881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:11.708369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a0c63-2760-4380-a7be-406f8df86db4 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:11.727752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 752/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:13.682492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:13.683053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:13.683053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:13.684434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 733/1485] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:13.689684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276fcd-e804-4b7d-b359-977df80287a0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:13.690109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276fcd-e804-4b7d-b359-977df80287a0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:13.690234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276fcd-e804-4b7d-b359-977df80287a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:13.755615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-76276fcd-e804-4b7d-b359-977df80287a0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:13.756816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-76276fcd-e804-4b7d-b359-977df80287a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:13.853109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 753/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:14.139587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:14.141159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:14.244224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4336-79ed-4a19-8fbc-e7a380f59d0d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:14.315699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:14.317577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:14.318179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:14.318457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:14.318728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:14.320334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:14.429156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:14.431057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e725e-3c41-4b88-99e8-9ef2c0f820ee service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:14.449473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 734/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:16.451119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:16.451702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:16.451702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:16.453032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 754/1488] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:16.458124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768905a-f20d-43ba-8e6a-12ed0348381c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:16.458539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768905a-f20d-43ba-8e6a-12ed0348381c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:16.458683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768905a-f20d-43ba-8e6a-12ed0348381c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:16.526148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f768905a-f20d-43ba-8e6a-12ed0348381c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:16.527392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f768905a-f20d-43ba-8e6a-12ed0348381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:16.581435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:16.626080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 735/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:16.917617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:16.919148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:16.954579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:17.021285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ad5ab-8545-419b-8f99-994d718f4dd9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:17.093032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:17.093625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:17.093898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:17.094164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:17.095750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:17.173958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:17.203857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:17.205312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7ceb-f451-45e0-8568-9aca0f7d8d13 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:17.224084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 755/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:19.488524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:19.489080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:19.489080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:19.490426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 736/1491] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:05:19.496305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c89ac-d142-416d-b5c9-0cb76b4d83f0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:19.496710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c89ac-d142-416d-b5c9-0cb76b4d83f0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:19.496844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c89ac-d142-416d-b5c9-0cb76b4d83f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:19.564269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-717c89ac-d142-416d-b5c9-0cb76b4d83f0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:05:19.565460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-717c89ac-d142-416d-b5c9-0cb76b4d83f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:19.664592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 756/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 09:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:05:19.960635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:05:19.962531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:20.068406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ca9c9-7005-4471-8ed2-8553d60e3e3f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:05:20.143935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:05:20.144532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:05:20.144811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:05:20.145092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:05:20.146694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:05:20.156212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:20.256019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:05:20.257372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a65b6-3346-4c3b-b2fb-88a643a58dd4 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:05:20.275791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 737/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:05:29.661760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=82 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:05:30.272591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=50 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:42.072336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:42.072928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:42.072928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:42.148428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:06:42.159686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:42.246653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 757/1494] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:06:42.255954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:42.256453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:42.256559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:42.335373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:06:42.336497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:42.434190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 738/1495] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:06:42.439005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:42.440293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:42.452355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88881721-200c-4eba-adc3-8929ae38ab87 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:42.514408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:42.549257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:42.549859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:42.550131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:42.550396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:42.551030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:06:42.551356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:42.552376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:42.552376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:06:42.552376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:06:42.553269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:42.565912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 758/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:06:42.574270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:42.583464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301361a5-70b5-4106-99ce-8bdc26aab043 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:42.644318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:42.678941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:42.679575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:42.679855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:42.680128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:42.680760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:42.682554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:42.682554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:06:42.682554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:06:42.683792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:42.819285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 739/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:06:42 2022] POST /identity/v3/projects => generated 377 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:06:42.823046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:42.824639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:42.859673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeeadae-b086-4022-89c4-49d32b50bf1f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:42.952492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:42.953004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:42.953456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:42.953732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:42.954437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:42.955472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:42.955472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:06:42.955472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:06:42.956676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:43.058698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 759/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:06:42 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:06:43.063725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:43.065232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:43.127251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e714a1-cc49-492f-8012-91f1fafdb0a3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:43.212842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:43.223423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:43.224029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.224317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.224588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.225253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:43.226325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:43.226325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:06:43.226325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:06:43.227309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:43.240552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 740/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:06:43.244632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:43.246151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:43.258106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5c162-e55b-4f48-96f9-3cf44d87e21f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:43.283575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:43.353216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:43.353819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.354095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.354364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.354994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:43.356029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:43.356029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:06:43.356029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:06:43.357442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da2be3-8848-4b37-a3a2-e4f02ad9d1c5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:43.369563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 760/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:06:43.374469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:43.376017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:43.409090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634da02-68a8-4399-9c12-8460bd2cc40c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:43.503126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:43.503715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.503992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.504264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] PATH_INFO: `/v3/projects/a2e39ceb59ab4f39ac03ddce843b0742/users/4606a209c47a4b34a3060664a3146cae/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.526598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e39ceb59ab4f39ac03ddce843b0742, user_id=4606a209c47a4b34a3060664a3146cae, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:43.527919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:43.527919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:06:43.527919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:06:43.530412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b4375-f08d-4cb0-8368-21a0a92cbdac admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:43.624059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 741/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:06:43 2022] PUT /identity/v3/projects/a2e39ceb59ab4f39ac03ddce843b0742/users/4606a209c47a4b34a3060664a3146cae/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:06:43.636379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd59c1-a094-4f96-9e0e-79b502ef67e0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.636667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd59c1-a094-4f96-9e0e-79b502ef67e0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.636943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd59c1-a094-4f96-9e0e-79b502ef67e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.694299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3ebd59c1-a094-4f96-9e0e-79b502ef67e0 None None] MFA Rules not processed for user `4606a209c47a4b34a3060664a3146cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:06:43.695509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3ebd59c1-a094-4f96-9e0e-79b502ef67e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:43.770975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 761/1502] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:06:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:06:43.784770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.785057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.785333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.786680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 742/1503] 10.222.0.46 () {54 vars in 964 bytes} [Thu Sep 1 09:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:06:43.791411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e530d-e407-4223-8d43-1a85542c7e1c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:43.791696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e530d-e407-4223-8d43-1a85542c7e1c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:43.791987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e530d-e407-4223-8d43-1a85542c7e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:43.823753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:43.869157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-124e530d-e407-4223-8d43-1a85542c7e1c None None] MFA Rules not processed for user `33f0625dda614821a5d9ca6c75de33d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:06:43.870360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-124e530d-e407-4223-8d43-1a85542c7e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:43.946337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 762/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Sep 1 09:06:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:06:43.956679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:43.958197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:44.007162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:44.061621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4338d-9cbb-494a-a78c-64350a4b040f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:44.134723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:44.135354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:44.135635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:44.135908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:44.137511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:44.236133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:44.237569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:44.241091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:44.255071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 743/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:06:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:06:44.301089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633950-04a3-4f2c-8a50-9b51c52a2b3c None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:44.301555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633950-04a3-4f2c-8a50-9b51c52a2b3c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:44.301665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633950-04a3-4f2c-8a50-9b51c52a2b3c None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:44.302873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 763/1506] 10.222.0.46 () {52 vars in 830 bytes} [Thu Sep 1 09:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:06:44.306363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:44.307590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:44.405782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1d75c-c41a-4217-99a7-b1b66e343ba8 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:44.499023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:44.499678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:44.499957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:44.500229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] PATH_INFO: `/v3/users/4606a209c47a4b34a3060664a3146cae` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:44.512505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4606a209c47a4b34a3060664a3146cae)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:44.514047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:44.514047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:06:44.514047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:06:44.515322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:44.524861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:44.752059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4606a209c47a4b34a3060664a3146cae'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:44.894692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4606a209c47a4b34a3060664a3146cae'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:44.983510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4606a209c47a4b34a3060664a3146cae'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:44.983963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4606a209c47a4b34a3060664a3146cae'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:44.984452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4606a209c47a4b34a3060664a3146cae'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:45.003715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 744/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:06:44 2022] DELETE /identity/v3/users/4606a209c47a4b34a3060664a3146cae => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:06:45.008057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:45.008334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:45.008605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:45.065363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:06:45.066547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:45.141085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:45.153814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 764/1508] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:06:45.164048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:45.165276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:45.263743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31ab71-d6c2-4a36-9f59-5ac31e424a73 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:45.316591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:45.334906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:45.335564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:45.335842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:45.336112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:45.337705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:45.446553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:45.448034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d8deb-6ee7-41f8-bbcf-52cf8cdca2c5 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:45.467672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 745/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:06:45.986043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:06:45.989025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:06:45.999028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d82b2f-925b-404d-85c7-12643d592d0e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:06:46.098200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:06:46.098741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:06:46.099011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:06:46.099307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] PATH_INFO: `/v3/projects/a2e39ceb59ab4f39ac03ddce843b0742` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:06:46.109693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e39ceb59ab4f39ac03ddce843b0742)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:06:46.110703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:06:46.110703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:06:46.110703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:06:46.111940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:06:46.221323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e39ceb59ab4f39ac03ddce843b0742', 'request_id': 'req-f8447ee3-56d6-4214-8044-21e02a56579c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:46.295698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e39ceb59ab4f39ac03ddce843b0742', 'request_id': 'req-f8447ee3-56d6-4214-8044-21e02a56579c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:46.309228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e39ceb59ab4f39ac03ddce843b0742', 'request_id': 'req-f8447ee3-56d6-4214-8044-21e02a56579c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:46.332274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] The token cache is being invalidate because project a2e39ceb59ab4f39ac03ddce843b0742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:06:46.333812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:46.333812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f8447ee3-56d6-4214-8044-21e02a56579c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:06:46.334459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 765/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:06:45 2022] DELETE /identity/v3/projects/a2e39ceb59ab4f39ac03ddce843b0742 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:06:55.464666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=62 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:06:56.330286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:08:13.055580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:13.056167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:13.056167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:13.057458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:13.062187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eeb2-7a71-41bc-9c94-f1154bbe1243 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:13.062620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eeb2-7a71-41bc-9c94-f1154bbe1243 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:13.062723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505eeb2-7a71-41bc-9c94-f1154bbe1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:13.145913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3505eeb2-7a71-41bc-9c94-f1154bbe1243 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:13.147195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3505eeb2-7a71-41bc-9c94-f1154bbe1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:13.250744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:13.407527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:13.409781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:13.531356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1ee9b-b660-48ac-9953-6f976c9f70b5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:13.537314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:13.537915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:13.538191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:13.538466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:13.546704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:13.548395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cdcdd-2061-4ad7-911c-0abdbe5bfc10 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:13.558791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 747/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:14.448967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07653f-673c-45b9-9b9e-cf13bf5d8846 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:14.449559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07653f-673c-45b9-9b9e-cf13bf5d8846 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:14.449559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07653f-673c-45b9-9b9e-cf13bf5d8846 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:14.450837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:14.455697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755f57b-c6dd-473c-b89a-9c49c622f063 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:14.456147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755f57b-c6dd-473c-b89a-9c49c622f063 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:14.456261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755f57b-c6dd-473c-b89a-9c49c622f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:14.477201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4755f57b-c6dd-473c-b89a-9c49c622f063 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:14.478386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4755f57b-c6dd-473c-b89a-9c49c622f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:14.486374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:14.637068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07653f-673c-45b9-9b9e-cf13bf5d8846 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:14.652136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07653f-673c-45b9-9b9e-cf13bf5d8846 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:14.657801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:14.658390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:14.658671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:14.658944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:14.667576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:14.669160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe931fe-35cc-49e7-a7e1-39da7ff57b43 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:14.678618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 768/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:15.301914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e160e-b651-4b32-a3f5-d10d6a6a712a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:15.302194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e160e-b651-4b32-a3f5-d10d6a6a712a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:15.302467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e160e-b651-4b32-a3f5-d10d6a6a712a None None] PATH_INFO: `/v3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:15.304385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 749/1517] 10.222.0.55 () {54 vars in 925 bytes} [Thu Sep 1 09:08:15 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:08:15.310559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d4b53-8d93-42d5-9ca7-98611153fbbf None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:15.310844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d4b53-8d93-42d5-9ca7-98611153fbbf None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:15.311139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d4b53-8d93-42d5-9ca7-98611153fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:15.370893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f73d4b53-8d93-42d5-9ca7-98611153fbbf None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:15.372133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f73d4b53-8d93-42d5-9ca7-98611153fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:15.389381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:08:15.442852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 769/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Sep 1 09:08:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:24.471441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=56 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:08:24.531019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b7c7-33c4-4929-81ff-3c29abfe1ab4 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:24.531330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b7c7-33c4-4929-81ff-3c29abfe1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:24.531845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b7c7-33c4-4929-81ff-3c29abfe1ab4 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:24.533236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 750/1519] 10.222.0.46 () {54 vars in 961 bytes} [Thu Sep 1 09:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:24.538170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d06f3-c5aa-4052-bfaf-202a51080881 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:24.538670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d06f3-c5aa-4052-bfaf-202a51080881 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:24.538769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d06f3-c5aa-4052-bfaf-202a51080881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:24.559554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d06d06f3-c5aa-4052-bfaf-202a51080881 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:24.560756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d06d06f3-c5aa-4052-bfaf-202a51080881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:24.569298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 770/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Sep 1 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:24.576767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1b7c7-33c4-4929-81ff-3c29abfe1ab4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:24.590243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1b7c7-33c4-4929-81ff-3c29abfe1ab4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:24.595024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:24.595650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:24.595926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:24.596197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:24.605530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:24.607118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f707d8-7d51-41fa-aaca-27c0d2142df8 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:24.616561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 751/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:25.912782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15603e-d161-4107-b089-5cb6ec3643c6 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:25.913332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15603e-d161-4107-b089-5cb6ec3643c6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:25.913332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15603e-d161-4107-b089-5cb6ec3643c6 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:25.915280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:25.920112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1eb3e-8e4c-4d0e-afd6-47a2ad3b0bc3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:25.920536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1eb3e-8e4c-4d0e-afd6-47a2ad3b0bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:25.920644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1eb3e-8e4c-4d0e-afd6-47a2ad3b0bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:25.940370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f7b1eb3e-8e4c-4d0e-afd6-47a2ad3b0bc3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:25.941549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f7b1eb3e-8e4c-4d0e-afd6-47a2ad3b0bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:25.949416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:26.103253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15603e-d161-4107-b089-5cb6ec3643c6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:26.117556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15603e-d161-4107-b089-5cb6ec3643c6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:26.123015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:26.123632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:26.123920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:26.124198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:26.132210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:26.134597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f31cd-9977-49e2-8e7e-c23b4277a965 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:26.144394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 772/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:26.957162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19307-d104-4266-8001-8251846bb55c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:26.957720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19307-d104-4266-8001-8251846bb55c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:26.957720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19307-d104-4266-8001-8251846bb55c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:26.959034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 09:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:26.964317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c0b0-a340-45c3-9fe2-0a8754a8d247 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:26.964721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c0b0-a340-45c3-9fe2-0a8754a8d247 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:26.964833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865c0b0-a340-45c3-9fe2-0a8754a8d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:26.984330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9865c0b0-a340-45c3-9fe2-0a8754a8d247 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:26.985502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9865c0b0-a340-45c3-9fe2-0a8754a8d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:26.994247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:27.144818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19307-d104-4266-8001-8251846bb55c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:27.159947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19307-d104-4266-8001-8251846bb55c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:27.165437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:27.166052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:27.166331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:27.166602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:27.174776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:27.176396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73846de2-b038-4f10-8865-2875e73ed12e service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:27.185626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 754/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:30.831360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:30.831960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:30.831960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:30.833173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:30.838310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a62e-d3fb-48f2-8011-ac355edb15f8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:30.838709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a62e-d3fb-48f2-8011-ac355edb15f8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:30.838849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92a62e-d3fb-48f2-8011-ac355edb15f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:30.857895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3f92a62e-d3fb-48f2-8011-ac355edb15f8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:30.859066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3f92a62e-d3fb-48f2-8011-ac355edb15f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:30.866712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:31.157482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:31.159228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:31.238193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f6b13-e08f-4de3-b6d3-202ee82989bc None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:31.241847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:31.243196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:31.243613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:31.243893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:31.252807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:31.254234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e987259-1346-42e8-9e5f-a8f20730f787 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:31.264435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 775/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:33.179316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02cf5-cf73-4be6-844f-9fd797eb908a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:33.180375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02cf5-cf73-4be6-844f-9fd797eb908a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:33.180375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02cf5-cf73-4be6-844f-9fd797eb908a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:33.182060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:33.187320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3308-ece5-4d47-b502-706bc2616940 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:33.187771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3308-ece5-4d47-b502-706bc2616940 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:33.187880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca3308-ece5-4d47-b502-706bc2616940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:33.208440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e2ca3308-ece5-4d47-b502-706bc2616940 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:33.209617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e2ca3308-ece5-4d47-b502-706bc2616940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:33.217776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:33.504548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02cf5-cf73-4be6-844f-9fd797eb908a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:33.516141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02cf5-cf73-4be6-844f-9fd797eb908a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:33.519661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:33.520299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:33.520584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:33.520886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:33.529820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:33.531279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab78b6d-eaa1-4c05-b913-fbf7fb1bb895 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:33.540800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 757/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:36.984775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:36.985308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:36.985308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:36.986550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:36.991714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d42f-ff26-4d43-b288-4b4ad1dd8b5a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:36.992193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d42f-ff26-4d43-b288-4b4ad1dd8b5a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:36.992303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d42f-ff26-4d43-b288-4b4ad1dd8b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:37.012451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7df9d42f-ff26-4d43-b288-4b4ad1dd8b5a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:37.013635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7df9d42f-ff26-4d43-b288-4b4ad1dd8b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:37.021352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:37.571014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:37.572872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:37.637453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:08:37.650295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031c1e-fa23-4f33-a0df-e372d62519b7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:37.653777 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:37.654372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:37.654642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:37.654907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:37.663887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:37.665896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b031901-9561-47ff-b54e-097b9c5458b7 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:37.675525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 778/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:39.621072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735b4-c074-4670-8c75-46d81421e474 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:39.622086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735b4-c074-4670-8c75-46d81421e474 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:39.622580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993735b4-c074-4670-8c75-46d81421e474 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:39.624475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 09:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:08:39.630061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d6c9-731e-4ce9-aad3-fb0b0135c5b2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:39.630557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d6c9-731e-4ce9-aad3-fb0b0135c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:39.630680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d6c9-731e-4ce9-aad3-fb0b0135c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:39.650008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6d05d6c9-731e-4ce9-aad3-fb0b0135c5b2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:08:39.651221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6d05d6c9-731e-4ce9-aad3-fb0b0135c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:39.659156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 09:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:08:40.221622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993735b4-c074-4670-8c75-46d81421e474 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:08:40.223326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-993735b4-c074-4670-8c75-46d81421e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:08:40.236844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993735b4-c074-4670-8c75-46d81421e474 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:08:40.240305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:08:40.240882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:08:40.241157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:08:40.241425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:08:40.250172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:08:40.251614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150990b-bf7c-4c83-93aa-29ced7d8f712 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:08:40.261000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 760/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:08:49.645062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=11 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:08:50.256767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=33 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:04.211712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.211712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.211712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.236489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.237876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.250500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 780/1540] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.259233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.260216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.260705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.289962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.291565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.304556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 761/1541] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.309925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.322861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a52ab-6dcb-446b-a1aa-ee096a1136dd None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.331600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.332228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.332509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.332782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.333427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:04.333731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.335358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a936abc-3059-414e-9e9e-212dc89436c2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.356861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 781/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:04.365323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.375117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372978-01bf-4fc5-8d32-e460b297c25c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.375117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372978-01bf-4fc5-8d32-e460b297c25c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.375117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372978-01bf-4fc5-8d32-e460b297c25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.380283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16fb4e-5634-4209-b91a-7d8b4206ff25 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.385381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.386410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.386966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.389636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.391598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.393765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.412372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fa372978-01bf-4fc5-8d32-e460b297c25c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.413620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fa372978-01bf-4fc5-8d32-e460b297c25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.448763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 782/1543] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:04.464813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.465013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.465294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.508631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.510827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.520302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 762/1544] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:04.528185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.529769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 783/1545] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.535044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.559900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ab05-e8a0-44ed-8703-9f7b3e79ba14 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.673905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e0f9-3159-462a-9434-2a7f5d1e2c53 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.686583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:10:04.690999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:10:04.691596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc8b7-0d6a-4210-8dab-45b77423484c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.741450 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.742005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.742501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.742788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.743510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:04.743806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.745438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e036d9a-236b-4fe6-8bf4-b7d51971ce7d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.759831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 784/1546] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:04.761245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00f90-0661-4acd-bfc1-a2f86f4e2507 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.761544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00f90-0661-4acd-bfc1-a2f86f4e2507 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.762393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d00f90-0661-4acd-bfc1-a2f86f4e2507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.784099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a6d00f90-0661-4acd-bfc1-a2f86f4e2507 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.787560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a6d00f90-0661-4acd-bfc1-a2f86f4e2507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.790779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 763/1547] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:04.792081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.792510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.792789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.797645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 785/1548] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.798882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc623ae4-fc62-4a9a-a972-0308b9a94474 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.799200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc623ae4-fc62-4a9a-a972-0308b9a94474 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.799515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc623ae4-fc62-4a9a-a972-0308b9a94474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.818838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.820148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.828578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bc623ae4-fc62-4a9a-a972-0308b9a94474 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.829541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 764/1549] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.829779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bc623ae4-fc62-4a9a-a972-0308b9a94474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.830309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.841140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878806f-ad17-4073-8a4a-645f022985ce None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.844558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 786/1550] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.846219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.846721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.847073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.847498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.847677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.847954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.848228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.848856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.850914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675e2e-8cd1-4ca5-b21c-5d9504838871 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.868912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.870115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.878639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 787/1551] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.879453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:04.889407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b95e2a-d83e-4d0b-9030-54e01217d647 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:04.893981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:04.894563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.894844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.895143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.895809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:04.897400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a500bd4-1723-462a-9854-8c7c572f431e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:04.907297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 788/1552] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:04.908424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daccd4-a43c-4270-b2bf-bef10aef614b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.908709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daccd4-a43c-4270-b2bf-bef10aef614b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.908984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25daccd4-a43c-4270-b2bf-bef10aef614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.927962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-25daccd4-a43c-4270-b2bf-bef10aef614b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.929148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-25daccd4-a43c-4270-b2bf-bef10aef614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.937660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 789/1553] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:04.938790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc441-05fe-4569-8e76-40e2bc0edf46 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.939071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc441-05fe-4569-8e76-40e2bc0edf46 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.939375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc441-05fe-4569-8e76-40e2bc0edf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.958555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f69fc441-05fe-4569-8e76-40e2bc0edf46 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.959765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f69fc441-05fe-4569-8e76-40e2bc0edf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:04.961888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 765/1554] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:04.963168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b39a-4acd-4ce6-a1b0-f5214b018589 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:04.964046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b39a-4acd-4ce6-a1b0-f5214b018589 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:04.964046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b39a-4acd-4ce6-a1b0-f5214b018589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:04.982721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2fa1b39a-4acd-4ce6-a1b0-f5214b018589 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:04.983949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2fa1b39a-4acd-4ce6-a1b0-f5214b018589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.027421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 790/1555] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.028910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0aef2-042a-4fc9-a733-1343988970ba None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.029199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0aef2-042a-4fc9-a733-1343988970ba None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.029472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0aef2-042a-4fc9-a733-1343988970ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.049714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 766/1556] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.050049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ecc0aef2-042a-4fc9-a733-1343988970ba None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.051234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.051377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ecc0aef2-042a-4fc9-a733-1343988970ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.051559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.051782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.060789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 791/1557] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.061998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.062281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.062553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.072770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.074093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.081622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.082280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 767/1558] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.082807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.083270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.091022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 792/1559] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.091700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.092928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575019-655e-4bfe-99ed-9cc9fc3a7b09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.097745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.098326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.098609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.098877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.099538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.100779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba56314-c225-48bd-8f7f-a0acadf145e0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.101117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.105297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.106419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.106842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.107149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.107809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.108111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.109694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684bbec-82fe-4661-842a-c282fe71e6d4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.111445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 768/1560] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:05.115498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.116782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:05.123901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 793/1561] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.130269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.130269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.130269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.130568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5f27c-d625-4e4e-9611-fd08995f47cb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.131770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.132734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.133859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.134144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.134772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.137552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.144440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.145640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.145868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:05.154996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 794/1562] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.157758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.165299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1515d4-58f8-470b-8e3d-8bd27119b6af None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.169958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.170535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.170814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.171085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.171744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.172062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.173633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578316f3-27dd-4f53-93a6-513fc27138ec admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.185605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 795/1563] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.187029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.187344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.187619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.207437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.208612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.217445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 796/1564] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.218233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.227607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7df88-036d-43bb-b830-cab823702b8a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.232592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.233174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.235060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.235060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.235060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.235060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.236276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.247731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 797/1565] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.248502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.258175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0e68-5975-4715-99a0-0c17b09dedd7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.260129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 769/1566] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.265293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.265422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.265546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.265760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.266041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.266668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.266972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.268687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.271048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc727-44dc-4893-b6d8-1ec0ae6954c7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.275668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.276344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.276618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.276890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] PATH_INFO: `/v3/projects/e27cafb5f07e449295c45c033ee62256/users/f6d6f82f9aa6447591cdd55d2a040ea4/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.282187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 798/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.283024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.293187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27cafb5f07e449295c45c033ee62256, user_id=f6d6f82f9aa6447591cdd55d2a040ea4, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.296100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.298528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12351054-8547-4e99-991c-6c8191d8011c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.301416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.302035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.302315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.302584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.303242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.304241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:05.304241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:10:05.304241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:10:05.305434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.367512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 770/1568] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:05 2022] PUT /identity/v3/projects/e27cafb5f07e449295c45c033ee62256/users/f6d6f82f9aa6447591cdd55d2a040ea4/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:05.368476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.385880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 799/1569] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.387269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.432229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a3de-c15a-4e81-b463-8cbb6c6122d2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.453814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ff779-ee2d-4f33-9a95-329f3c713e77 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.458477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.459078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.459383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.459655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.460302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.462203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.491568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.492182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.492457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.492727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.493387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.493688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.494689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:05.494689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:10:05.494689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:10:05.495607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.507202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 771/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.507987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.517695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06fa1c-7340-4dbb-90dd-e84765aecefa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.522354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.522918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.523232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.523505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.524116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:05.524409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.525961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.543610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 772/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:05.543610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.547910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4e07b-7ea4-4193-a47c-e3886a5d4366 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.552694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.553286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.553561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.553830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.554543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.556406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa06fd-26ba-425a-a96e-f997acf647e2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.576673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 800/1572] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.577517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.638443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929878-f3b8-4361-9d27-15c0b54da94c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.643066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.643683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.643964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.644233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.644858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.646529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.656672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 801/1573] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:05.657440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.663508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:05.667007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebea26-c662-424b-9607-4bcb8dadee16 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.671913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.676347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.676347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.676347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.676347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.676347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.678416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 773/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.679683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.680151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.680487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.700751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:05.702071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:05.727329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:05.759453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 774/1575] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:05.760250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.770801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf037e-5a25-4986-8dff-02c6e1d501d8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.773441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 802/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.774250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.778775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.780188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.780188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.780188 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.780772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.782753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.839237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff327-38ed-4ac3-bf8e-4e78cb4aa6ca admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.843840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.844436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.845258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.845258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.845575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.847523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.898883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 775/1577] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.899895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:05.910257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae237a6-bec8-4325-b0f9-19aee57a9d0a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:05.915224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:05.915804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:05.916087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:05.916363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:05.916979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:05.918939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:05.956369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 803/1578] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 389 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:05.957195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.018409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a05f-0501-4f7e-94a9-3cec67d561a4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.023046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.023667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.023947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.024219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.024834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.026045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 776/1579] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.027064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.030809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.087669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6348c-0f34-40f3-9eb3-4a56dbb6dafc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.092792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.093327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.093601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.093869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.094522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.096141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.105552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 777/1580] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.106365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.115516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98795-1990-456d-833f-2b084603aeb6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.117620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 804/1581] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:05 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.121447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.121588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.121588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.121910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.122246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.122859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.125142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.177394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5ca4-8aa8-4f42-a5d7-6aebff9023da admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.186858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:06.194147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 778/1582] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:06 2022] POST /identity/v3/users => generated 293 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.194147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.201823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8ee82-e93b-4162-a87d-65c73850d3c3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.206499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.207052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.207363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.207630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.208263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.210178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.227303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.227907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.228186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.228457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.238088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.239557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.249132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 805/1583] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:06.249918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.258635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516769e1-95e0-4163-93dc-1c1d85e1e240 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.263755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.264343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.264904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.264904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.265548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.267173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.278192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 806/1584] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.278804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.287032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 779/1585] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:06 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.287836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.288566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ab5e0-8b0e-4e83-a0d9-b5750a2cb711 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.294739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.295376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.295653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.295922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.296624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.298211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff73f4-027e-414f-b6be-27ed69a34a88 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.298639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.302886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.303547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.303826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.304099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.304716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.306743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.363587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 807/1586] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:06 2022] POST /identity/v3/users => generated 295 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.364239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.374366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7e76-c92a-459a-af06-0ebf4038352d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.379470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.380130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.380411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.380708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] PATH_INFO: `/v3/projects/f48655ade1f64ba7b8cfff5c9eb1f8de/users/fe2b892d49c84a3cb8515105d4639f61/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.396201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48655ade1f64ba7b8cfff5c9eb1f8de, user_id=fe2b892d49c84a3cb8515105d4639f61, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.397364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:06.397364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:10:06.397364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:10:06.399375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.423941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 780/1587] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:06 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:06.423941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.431236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91fe67-cb47-4ce7-8577-02efb83cd868 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.436694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.437264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.437546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.437814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.438442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.440075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.450833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 781/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.451655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.460788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33506-b62f-4933-8201-89f4fa685dcc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.465602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.466175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.466457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.466730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.467462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.468943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.475068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 808/1589] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:06 2022] PUT /identity/v3/projects/f48655ade1f64ba7b8cfff5c9eb1f8de/users/fe2b892d49c84a3cb8515105d4639f61/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:06.482591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.482713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 782/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.482713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.492537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb258d-fa11-46f6-a2b9-f874e6cb315c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.508442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:06.559716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989003d-3a13-41ca-a7e7-12790bbc45ec admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.565290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.566039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.566448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.566785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.567734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.569785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.577009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.577628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.577911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.578173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.578823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.580486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.584514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 809/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.588351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.592964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 783/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.593822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.600473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb240f9-9dca-48cb-a46a-a4ac21ae6d7b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.603602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a5110-46bf-4409-ad0c-b372a8d9fc1a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.606073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.606682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.607015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.607325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.607969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.609759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.609882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.610553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.610833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.611112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.611786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.613374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.623196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 810/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.627322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.627462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 784/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.627462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.634466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec01249-8f5c-4f8e-96c6-2220d0c468da admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.639895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.640395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7851572-56b9-4512-bc37-8a655f2ae9fc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.640515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.640758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.641085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.641654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.643289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2a934-913c-429a-a0f3-7613723ba2ed admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.645990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.646748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.647524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.647802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.648469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.650057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.654551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 811/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.656125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.656125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.656336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.661998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 785/1596] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.663243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.673933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c052d-8168-449d-b55f-0360e4fe25f3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.678630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.683349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.683349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.683349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] PATH_INFO: `/v3/projects/b85131a2dcda402a8f6aded76993838b/users/fc68ff87997c44e695ef3204f1d57471/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.686863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] MFA Rules not processed for user `fe2b892d49c84a3cb8515105d4639f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:06.688112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:06.696287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85131a2dcda402a8f6aded76993838b, user_id=fc68ff87997c44e695ef3204f1d57471, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.698965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.716897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:06.757050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 812/1597] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:06.757772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.768577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fd862-b4b7-4c4f-8a5f-f7a89229bc82 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.773267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.773810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.774067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.774340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.775005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.776636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.788185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 813/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.789167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.798953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb251d-e025-4834-aadc-3fb5add718e8 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.803794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.804394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.804730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.805005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.805633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.807912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.811209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 786/1599] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:06 2022] PUT /identity/v3/projects/b85131a2dcda402a8f6aded76993838b/users/fc68ff87997c44e695ef3204f1d57471/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:06.811209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.820767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 814/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.821917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.832335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711b78-3610-4fed-89bd-6fa5932be66c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.885693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959483-0866-47ae-98a7-7ea726031cf9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.891803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.892321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.892638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.892907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.893651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.895281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.900969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.901692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.901970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.902249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] PATH_INFO: `/v3/projects/8ceea16a8d9244f2907a2662436a8fda/users/c9da0e0fd0d64db49da1f65bff93ac36/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.907096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 787/1601] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:06.907898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:06.919855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e9eb-fedd-4942-ac72-1821bf50ef41 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:06.922156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ceea16a8d9244f2907a2662436a8fda, user_id=c9da0e0fd0d64db49da1f65bff93ac36, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.924519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:06.924906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:06.925163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:06.925439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:06.925707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] PATH_INFO: `/v3/projects/c1035740bd454149963828f12066b70d/users/02240dc931e140498d82c6d91efb60c8/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:06.941726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1035740bd454149963828f12066b70d, user_id=02240dc931e140498d82c6d91efb60c8, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:06.944547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.071768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 788/1602] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:06 2022] PUT /identity/v3/projects/c1035740bd454149963828f12066b70d/users/02240dc931e140498d82c6d91efb60c8/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:07.072890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.075558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 815/1603] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:06 2022] PUT /identity/v3/projects/8ceea16a8d9244f2907a2662436a8fda/users/c9da0e0fd0d64db49da1f65bff93ac36/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:07.086961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.117531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:07.142753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32a49-6356-4e7d-b4cf-4a0288215e55 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.160701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3c3d-3892-46ee-a67b-d40e8b5fd431 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.165707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.166367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.166645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.166913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] PATH_INFO: `/v3/projects/f5045a17a253414e9c231bcd0d7d24b8/users/004ab0c0b138457e9a5e9288d039b932/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.183289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5045a17a253414e9c231bcd0d7d24b8, user_id=004ab0c0b138457e9a5e9288d039b932, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.187622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19de53-98b2-4c54-9b96-fac4b7842af2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.207066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.207805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.208092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.208362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.208986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.210895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.264834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 816/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:07 2022] PUT /identity/v3/projects/f5045a17a253414e9c231bcd0d7d24b8/users/004ab0c0b138457e9a5e9288d039b932/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:07.266185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.266473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.266742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.288549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:07.296562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] MFA Rules not processed for user `fc68ff87997c44e695ef3204f1d57471`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.297884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.304921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 789/1605] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:07.304921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.363795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 817/1606] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.364631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.375048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8a3-6efd-484c-a5c7-10cbd8889f29 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.380756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.381232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.387875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.387875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] PATH_INFO: `/v3/projects/b8799a08d0684b5c8d9cd3f89c3016d7/users/3d931a379404433b9c3c690dd9beb83d/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.397812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8799a08d0684b5c8d9cd3f89c3016d7, user_id=3d931a379404433b9c3c690dd9beb83d, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.400477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c14673-40fe-40f0-89ff-1f6177e58e6b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.441408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864c435-fa60-4e2c-9a00-d913d13c8fe8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.446090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.446755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.447033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.447334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] PATH_INFO: `/v3/projects/15f1ee0d68f646bcad60cbf31c3c684d/users/81816508df7c431fa3b4b495d11ac101/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.464179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f1ee0d68f646bcad60cbf31c3c684d, user_id=81816508df7c431fa3b4b495d11ac101, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.465625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc3060-a99a-47e8-8b1a-2384fcf722d8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.505653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 790/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:07 2022] PUT /identity/v3/projects/b8799a08d0684b5c8d9cd3f89c3016d7/users/3d931a379404433b9c3c690dd9beb83d/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:07.507072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186266b-6ca4-4bfd-a900-a3d955f54036 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.507389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186266b-6ca4-4bfd-a900-a3d955f54036 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.507662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186266b-6ca4-4bfd-a900-a3d955f54036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.527302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c186266b-6ca4-4bfd-a900-a3d955f54036 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.529355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c186266b-6ca4-4bfd-a900-a3d955f54036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.563096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 818/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:07 2022] PUT /identity/v3/projects/15f1ee0d68f646bcad60cbf31c3c684d/users/81816508df7c431fa3b4b495d11ac101/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:07.568213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.568500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.568797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.577893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:07.592491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 791/1609] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.593938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.594295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.594499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.598323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] MFA Rules not processed for user `c9da0e0fd0d64db49da1f65bff93ac36`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.599714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.614546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.615671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.669927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 819/1610] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.670808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.684524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 792/1611] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.685548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.688790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:07.762958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f0470-8b0c-4bec-a80f-5b2007850e00 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.764500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317faee8-1e14-4def-8c04-33e0f4fad3ff None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.766792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.767459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.767735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.768074 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.769259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.769854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.770131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.770396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.771344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.773206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.835958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:07.840104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f9fa6-1f57-4971-a956-7e21408401df service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:07.850159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 793/1612] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:07.851406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1880ec-924f-4263-b6f2-ff25934bc4b3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.851688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1880ec-924f-4263-b6f2-ff25934bc4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.851958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1880ec-924f-4263-b6f2-ff25934bc4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.873959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4e1880ec-924f-4263-b6f2-ff25934bc4b3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.875485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4e1880ec-924f-4263-b6f2-ff25934bc4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.883163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 794/1613] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.884416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.884777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.885103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.901292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 820/1614] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/users => generated 300 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:07.901292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.908222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:07.909692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.918619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 795/1615] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:07.919844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:07.934576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:07.977773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776d19a-0648-460a-b856-ef7a9de4d81f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:07.982720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:07.983381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:07.983745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:07.984111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:07.986268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:07.996940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.002209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a52d42-3882-4299-9293-164c2e2ce244 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.010713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 821/1616] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.012732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abee1-1542-4c6e-9f26-f70cca53e304 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.013236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abee1-1542-4c6e-9f26-f70cca53e304 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.013681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abee1-1542-4c6e-9f26-f70cca53e304 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.015334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 822/1617] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 09:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:08.016601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abee1-1542-4c6e-9f26-f70cca53e304 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.029834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abee1-1542-4c6e-9f26-f70cca53e304 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.032261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683252e-dc01-4885-a690-c8931ddb4d39 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.035137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.036869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.037359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.038006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.042059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.043181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.043181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.044109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.057581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.060792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.061118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.062353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.073447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 796/1618] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.073976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 823/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.075471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.075713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.085776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106fcd-d327-4d43-bb92-9f03f6a6c982 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.088138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f11b54-ed94-41dd-8518-43a4edb590c1 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.091038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.091743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.092848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.093218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.094585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.095482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.095832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.096214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.096923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.099118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899e0aa-b5be-42b1-bff7-6108ac1a9b1d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.105650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.107576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.111350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 797/1620] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:08.112918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.113266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.113609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.120543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 824/1621] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.121793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.133288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552026f9-bbad-471a-bf5e-462b7ae046ab service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.140238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.140966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.141249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.141523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] PATH_INFO: `/v3/projects/4b2be626a7704aab8e491d39a68b04df/users/c4ead43d7e9049b9ae3f1f844d0cf47e/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.162537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2be626a7704aab8e491d39a68b04df, user_id=c4ead43d7e9049b9ae3f1f844d0cf47e, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.165838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.175537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:08.176850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:08.248031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 825/1622] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:08 2022] PUT /identity/v3/projects/4b2be626a7704aab8e491d39a68b04df/users/c4ead43d7e9049b9ae3f1f844d0cf47e/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:08.252109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.262779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 798/1623] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:08.272294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.283988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b2986-58ec-4720-9168-0b6efefb416f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.334568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:08.344864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ca15d-e3eb-4e59-a8a8-20a508e28cd2 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.359141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.359467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.360623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.360623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.361613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.365762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.393569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 826/1624] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:08.397013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.417130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc769e-cd4b-41a9-85a9-318b89cd2b20 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.420058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.432749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.432749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.432749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.433112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.433112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.433112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.433112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] PATH_INFO: `/v3/projects/4b2be626a7704aab8e491d39a68b04df/users/c4ead43d7e9049b9ae3f1f844d0cf47e/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.433471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2be626a7704aab8e491d39a68b04df, user_id=c4ead43d7e9049b9ae3f1f844d0cf47e, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.433471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd51f7-71cb-47b5-9905-fcb1cee72baa admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.462449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 827/1625] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:08 2022] PUT /identity/v3/projects/4b2be626a7704aab8e491d39a68b04df/users/c4ead43d7e9049b9ae3f1f844d0cf47e/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:08.472806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e617c4-65a4-4d2c-b2a1-0c4beeb758b0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.473095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e617c4-65a4-4d2c-b2a1-0c4beeb758b0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.473375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e617c4-65a4-4d2c-b2a1-0c4beeb758b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.487501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:08.510009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.512010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.524013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 799/1626] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.533434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-18e617c4-65a4-4d2c-b2a1-0c4beeb758b0 None None] MFA Rules not processed for user `c4ead43d7e9049b9ae3f1f844d0cf47e`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:08.534831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-18e617c4-65a4-4d2c-b2a1-0c4beeb758b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:08.688897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 828/1627] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:08.698050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:08.774846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8e7ae-d8f6-4c10-a5bf-f5755e20e405 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:08.869821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d2b6-1e68-4f28-a083-ed4059c7a021 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.870104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d2b6-1e68-4f28-a083-ed4059c7a021 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.870371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058d2b6-1e68-4f28-a083-ed4059c7a021 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.872661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 829/1628] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 09:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:08.878049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.878336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.878609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.883669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:08.884643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:08.885900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:08.885900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:08.900144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:08.902318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:08.913506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:08.914775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:08.916971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 800/1629] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:08.957035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:09.064800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 830/1630] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:10:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:09.074191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:09.086749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff846712-1387-4175-af49-d22518e0fa62 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:09.097566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dedad7-ed17-478f-a13f-a227ca60f30a service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:09.099257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:09.100083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:09.111067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:09.112070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:09.112070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:09.112484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] PATH_INFO: `/v3/users/fe2b892d49c84a3cb8515105d4639f61` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:09.115555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe2b892d49c84a3cb8515105d4639f61)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:09.116987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:09.117464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:09.117600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:09.133014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:09.134853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf613a5-5986-4b97-9052-e247819bb3fd service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:09.218022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 831/1631] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:09.554892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:09.569901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe2b892d49c84a3cb8515105d4639f61'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:09.600700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe2b892d49c84a3cb8515105d4639f61'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:09.695002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe2b892d49c84a3cb8515105d4639f61'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:09.695965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe2b892d49c84a3cb8515105d4639f61'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:09.699819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe2b892d49c84a3cb8515105d4639f61'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:09.723026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 801/1632] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:08 2022] DELETE /identity/v3/users/fe2b892d49c84a3cb8515105d4639f61 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:09.724671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9028-e0ef-4c49-ab20-7fd11f3900dc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:09.725179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9028-e0ef-4c49-ab20-7fd11f3900dc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:09.725278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f9028-e0ef-4c49-ab20-7fd11f3900dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:09.755868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fb5f9028-e0ef-4c49-ab20-7fd11f3900dc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:09.756339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fb5f9028-e0ef-4c49-ab20-7fd11f3900dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:09.853557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:09.900311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 832/1633] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:09.917932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:09.919521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:10.018908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cfa54-c1a7-4ee9-945e-8bf613f0a8b7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:10.027728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:10.028698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.029266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.029781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.047350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:10.048924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8313e21-feca-49ab-abde-24faab41a016 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:10.061170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 802/1634] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:10.583989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff25fa-6f95-4e75-95ab-5edd597b4870 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.584511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff25fa-6f95-4e75-95ab-5edd597b4870 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.584693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff25fa-6f95-4e75-95ab-5edd597b4870 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.586327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 833/1635] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 09:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e824f8-40d6-42e3-a90b-13c03d5dfb80 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e824f8-40d6-42e3-a90b-13c03d5dfb80 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e824f8-40d6-42e3-a90b-13c03d5dfb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-91e824f8-40d6-42e3-a90b-13c03d5dfb80 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-91e824f8-40d6-42e3-a90b-13c03d5dfb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:10.808658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 803/1636] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:10.810420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.810420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.810420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.810420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 834/1637] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 09:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:10.810994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 804/1638] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:10.819250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:10.821870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:10.927523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76335db4-24a0-4c2e-a1ef-68f91b997f3f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:10.932113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:10.932695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:10.932967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:10.933239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:10.973057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:10.984992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:11.014662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 835/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:12.017303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:12.017303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:12.038742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab34663-c456-488b-b510-6a58e1a02e86 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:12.046789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:12.047782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:12.048239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:12.048556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] PATH_INFO: `/v3/projects/f48655ade1f64ba7b8cfff5c9eb1f8de` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:12.050069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48655ade1f64ba7b8cfff5c9eb1f8de)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:12.051249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:12.051249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Sep 01 09:10:12.051249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: Sep 01 09:10:12.053025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:12.254240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:12.315082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48655ade1f64ba7b8cfff5c9eb1f8de', 'request_id': 'req-349b0613-c325-4c0b-8e85-3c0bc099fb50'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:12.338394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48655ade1f64ba7b8cfff5c9eb1f8de', 'request_id': 'req-349b0613-c325-4c0b-8e85-3c0bc099fb50'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:12.373963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48655ade1f64ba7b8cfff5c9eb1f8de', 'request_id': 'req-349b0613-c325-4c0b-8e85-3c0bc099fb50'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:12.501375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] The token cache is being invalidate because project f48655ade1f64ba7b8cfff5c9eb1f8de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:12.501950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:12.504296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:12.509468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 805/1640] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:12 2022] DELETE /identity/v3/projects/f48655ade1f64ba7b8cfff5c9eb1f8de => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:12.514424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:12.533099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f9be1-d7da-4ee6-89d4-e486f331e02a service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:12.642918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:12.658387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:12.659563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:12.660773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] PATH_INFO: `/v3/users/c4ead43d7e9049b9ae3f1f844d0cf47e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:12.666674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ead43d7e9049b9ae3f1f844d0cf47e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:12.673218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: WARNING py.warnings [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 09:10:12.673218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: warnings.warn(msg) Sep 01 09:10:12.673218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: Sep 01 09:10:12.678794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:12.975092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ead43d7e9049b9ae3f1f844d0cf47e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:13.008807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ead43d7e9049b9ae3f1f844d0cf47e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:13.028687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ead43d7e9049b9ae3f1f844d0cf47e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:13.038239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ead43d7e9049b9ae3f1f844d0cf47e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:13.038239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ead43d7e9049b9ae3f1f844d0cf47e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:13.130741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 836/1641] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:12 2022] DELETE /identity/v3/users/c4ead43d7e9049b9ae3f1f844d0cf47e => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:13.816712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:13.818549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:13.833662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b0613-c325-4c0b-8e85-3c0bc099fb50 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:13.974317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:13.975341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:13.975781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:13.976213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] PATH_INFO: `/v3/users/c9da0e0fd0d64db49da1f65bff93ac36` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:13.977543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9da0e0fd0d64db49da1f65bff93ac36)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:13.979743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:14.234683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9da0e0fd0d64db49da1f65bff93ac36'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.256581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9da0e0fd0d64db49da1f65bff93ac36'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.258519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:14.262556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:14.280408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:14.302280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec1af1-a7ec-46f7-92d6-6ccc98986249 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:14.310563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:14.319382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:14.319382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:14.319382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] PATH_INFO: `/v3/projects/4b2be626a7704aab8e491d39a68b04df` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:14.319382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2be626a7704aab8e491d39a68b04df)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:14.319382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:14.364371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9da0e0fd0d64db49da1f65bff93ac36'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.371352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9da0e0fd0d64db49da1f65bff93ac36'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.371952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3cf6b6c-69ee-46c2-8252-e50faa005fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9da0e0fd0d64db49da1f65bff93ac36'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.406567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 806/1642] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:13 2022] DELETE /identity/v3/users/c9da0e0fd0d64db49da1f65bff93ac36 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:14.410850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:14.411282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:14.412613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:14.453670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:14.455595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:14.524682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2be626a7704aab8e491d39a68b04df', 'request_id': 'req-1fcee171-68ea-4f76-98e1-f541980349e9'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.590675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2be626a7704aab8e491d39a68b04df', 'request_id': 'req-1fcee171-68ea-4f76-98e1-f541980349e9'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.607875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2be626a7704aab8e491d39a68b04df', 'request_id': 'req-1fcee171-68ea-4f76-98e1-f541980349e9'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.622159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] The token cache is being invalidate because project 4b2be626a7704aab8e491d39a68b04df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:14.622490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.627613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1fcee171-68ea-4f76-98e1-f541980349e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:14.629972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 837/1643] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:14 2022] DELETE /identity/v3/projects/4b2be626a7704aab8e491d39a68b04df => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:14.635245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aad6b-1a50-4d9d-86d4-434bd9c4d90d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:14.635904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aad6b-1a50-4d9d-86d4-434bd9c4d90d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:14.636376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aad6b-1a50-4d9d-86d4-434bd9c4d90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:14.681071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:14.703335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f14aad6b-1a50-4d9d-86d4-434bd9c4d90d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:14.704479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f14aad6b-1a50-4d9d-86d4-434bd9c4d90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:14.711873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 807/1644] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:14.757629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:14.759981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:14.888745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f42f3-381f-41cb-8441-fe709acb9676 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:14.927181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 838/1645] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:14.947382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:14.947691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:14.948077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:14.998029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:15.002080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:15.025860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.026503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.026784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.027058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.028571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 839/1646] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:15.033734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.057231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142319a3-c51e-4c16-a54e-c4d5bd81cb01 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.076257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.080353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.103727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.104629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.105110 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.108991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.109955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:15.110411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.114805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.134868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 808/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:15.144636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 840/1648] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:10:15.150643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.175905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22bc20-ecde-423c-b0a2-0ff45a814b40 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.198736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.199622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.200370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.200860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.201644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.206687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.338607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 809/1649] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:15 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:15.339885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.359684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20b420-d33e-45fb-b342-884c4c808a4e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.577608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.578673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.578673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.578880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.579412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.581350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.671997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:15.681892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 841/1650] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:15 2022] POST /identity/v3/users => generated 289 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:15.682744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.697061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d5edf-42ff-4f18-b95b-a2668c4c31f2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.704926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.708524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.708524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.708524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.708524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.709925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.744177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 810/1651] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:15.744555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.756955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981241a-35e2-48e7-b5da-e9a5a55b9c42 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.764886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.771540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.771540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.771540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.771540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.771540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3cfed-810d-4810-a964-94fdc53ef007 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.797498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:15.806005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 842/1652] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:15.808772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541bb36-0b87-4e40-8ddb-b066eeac91e0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:15.815068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:15.816013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.816441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.816872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] PATH_INFO: `/v3/projects/7bfd4decd234423a9e98bcc90986c656/users/4da98704680e4f748e7f582b5d88f68a/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:15.861974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bfd4decd234423a9e98bcc90986c656, user_id=4da98704680e4f748e7f582b5d88f68a, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:15.864909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:15.974537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 811/1653] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:15 2022] PUT /identity/v3/projects/7bfd4decd234423a9e98bcc90986c656/users/4da98704680e4f748e7f582b5d88f68a/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:15.985161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:15.985715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:15.989544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:16.073540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] MFA Rules not processed for user `4da98704680e4f748e7f582b5d88f68a`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:16.078373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:16.312619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 843/1654] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:16.317687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:16.442631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:16.464438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ce054-5efc-4aa2-bc8f-6a8ad158ac74 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:16.501551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:16.506472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:16.507959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:16.510827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:16.513860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:16.533070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:16.691881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 812/1655] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:16 2022] POST /identity/v3/projects => generated 379 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:16.693688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:16.709070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22d6d-53e5-45e2-af75-77c0f491cad3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:16.879202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:16.883738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:16.884259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:16.884735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:16.885617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:16.888751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:16.893532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:16.897272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:16.926193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73122e-c52b-43fc-9d41-31db06ee0b66 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:16.938973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:16.939839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:16.940772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:16.941209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] PATH_INFO: `/v3/projects/8ceea16a8d9244f2907a2662436a8fda` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:16.944884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ceea16a8d9244f2907a2662436a8fda)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:16.950435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.001133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 844/1656] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:16 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:17.004871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.025512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bed839-1772-4678-849e-ffcd2c37b7de admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.032098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.033036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.033489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.033913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.034775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.036701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.054870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 845/1657] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:17.056775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.074524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1988e8c-6795-4b9e-b28f-4508bb6b9146 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.080651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.081595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.082249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.083053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.084029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.086396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.105686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 846/1658] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:17.109577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.146979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d354-75df-4778-9ab4-6333fa00c807 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.151991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ceea16a8d9244f2907a2662436a8fda', 'request_id': 'req-e93e32d8-a0d2-4697-bb0c-814688b57cc8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:17.168085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ceea16a8d9244f2907a2662436a8fda', 'request_id': 'req-e93e32d8-a0d2-4697-bb0c-814688b57cc8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:17.180879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ceea16a8d9244f2907a2662436a8fda', 'request_id': 'req-e93e32d8-a0d2-4697-bb0c-814688b57cc8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:17.249258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:17.274232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.274232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.274232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.277509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] PATH_INFO: `/v3/projects/cb2528524b124fafb4af2344a804ec5b/users/41f934aa0c3d47d9af8b416b9053cf93/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.302864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] The token cache is being invalidate because project 8ceea16a8d9244f2907a2662436a8fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:17.303520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:17.305001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e93e32d8-a0d2-4697-bb0c-814688b57cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:17.307333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 813/1659] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:16 2022] DELETE /identity/v3/projects/8ceea16a8d9244f2907a2662436a8fda => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:17.312338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30bb15-f38a-4a77-b176-361ad1580b2f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.313468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30bb15-f38a-4a77-b176-361ad1580b2f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.314109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30bb15-f38a-4a77-b176-361ad1580b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.322165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2528524b124fafb4af2344a804ec5b, user_id=41f934aa0c3d47d9af8b416b9053cf93, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.329359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9015a-4d4a-4ec8-b2e3-2b39e7888243 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.371056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5f30bb15-f38a-4a77-b176-361ad1580b2f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:17.389044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5f30bb15-f38a-4a77-b176-361ad1580b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:17.397847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 814/1660] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:17.406991 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803e91-e492-42c9-9849-c486c967541d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.407935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803e91-e492-42c9-9849-c486c967541d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.408435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803e91-e492-42c9-9849-c486c967541d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.458834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-32803e91-e492-42c9-9849-c486c967541d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:17.461732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-32803e91-e492-42c9-9849-c486c967541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:17.481505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 847/1661] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:17 2022] PUT /identity/v3/projects/cb2528524b124fafb4af2344a804ec5b/users/41f934aa0c3d47d9af8b416b9053cf93/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:17.496562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.497171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.498207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.603995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] MFA Rules not processed for user `41f934aa0c3d47d9af8b416b9053cf93`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:17.608593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:17.723066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 815/1662] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:17.727024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32803e91-e492-42c9-9849-c486c967541d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.744943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32803e91-e492-42c9-9849-c486c967541d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.763092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.767864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.768014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.768285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.768960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:17.769262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.770915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.839948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 816/1663] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:17.846752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.872180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 848/1664] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:17.876286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:17.878691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:17.880370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18af3ee-97e3-4dc6-8757-88455c5c89b3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.900999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.901864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.902364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.903376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.908635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.908635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:17.925871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c26f0-9172-49fb-9769-128e1319c1aa None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:17.932596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:17.938423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:17.938423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:17.938423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:17.938423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:17.940337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.074434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 817/1665] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:17 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:18.076810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.145785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:18.160705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 849/1666] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:17 2022] POST /identity/v3/projects => generated 379 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:18.168223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.214204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6ed9-a0c7-444e-b400-cab169229814 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.422363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29513383-4acb-47c7-9d62-1b8cf0682428 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.440204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.443060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.443979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.444438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.444857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.449061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.453977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.453977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.453977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.454246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.459030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.459488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.584940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 850/1667] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:18 2022] POST /identity/v3/users => generated 296 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:18.585588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 818/1668] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:18 2022] POST /identity/v3/users => generated 283 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:18.589987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.591900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.614634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a90cf-db69-401c-b2e5-aacb867a19f9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.622735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.626556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.626556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.626556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.626556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.626961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d1a6-fac5-41f0-be97-95a942afc03e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.627605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.631476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.632383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.633009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.633434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.634312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.640304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.651792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 819/1669] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:18.660738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.681414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 851/1670] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:18.685007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.687588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504cf63-4bc5-457b-8ce7-d51b65bcab82 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.709815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a8a93-1e12-4a7d-aecc-644f22968f0a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.720508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.721491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.722082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.722463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.724009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.725973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.730153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.731220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.731673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.732407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] PATH_INFO: `/v3/projects/c6a405434bb443bc8c8f567d084744a1/users/be26427323464b0fbbfd40e7d67a4522/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.759155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 820/1671] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:18.764117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:18.766672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.774241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a405434bb443bc8c8f567d084744a1, user_id=be26427323464b0fbbfd40e7d67a4522, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.779575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c3af8-d29f-4814-b9d9-d626e1fc0a47 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:18.781430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.789090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:18.789776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.792863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.792863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] PATH_INFO: `/v3/projects/d82f55c2986043b08bfe8f562bbb9670/users/7211fb1a5e0e488086e5221582578c0e/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:18.868756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82f55c2986043b08bfe8f562bbb9670, user_id=7211fb1a5e0e488086e5221582578c0e, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:18.873266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d0004-7271-467a-8186-a678f68f27bb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:18.907853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 852/1672] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:18 2022] PUT /identity/v3/projects/c6a405434bb443bc8c8f567d084744a1/users/be26427323464b0fbbfd40e7d67a4522/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:18.911480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:18.982199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 821/1673] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:18 2022] PUT /identity/v3/projects/d82f55c2986043b08bfe8f562bbb9670/users/7211fb1a5e0e488086e5221582578c0e/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:18.991145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:18.992608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:18.993711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.092604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a528f5-c5d7-4ade-8a99-9420cb06effb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.097186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] MFA Rules not processed for user `7211fb1a5e0e488086e5221582578c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:19.099984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:19.256670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.258352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.259931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.260230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.260898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:19.265491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:19.291282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 822/1674] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:19.297748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 853/1675] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:19.298142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.302836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.317956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b648c-1ac8-45a3-b3ef-81b4ba1c94a5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.326801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.328166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.328851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.329313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.330162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:19.332607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:19.335921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45558-d152-4f8d-b29f-0aebdad72a64 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.346926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.347673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.350528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.350823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] PATH_INFO: `/v3/projects/c6a405434bb443bc8c8f567d084744a1/users/be26427323464b0fbbfd40e7d67a4522/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.355097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a405434bb443bc8c8f567d084744a1, user_id=be26427323464b0fbbfd40e7d67a4522, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:19.363027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfee1d-66d8-470e-aec0-366fd342384d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:19.387074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 854/1676] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:19 2022] PUT /identity/v3/projects/c6a405434bb443bc8c8f567d084744a1/users/be26427323464b0fbbfd40e7d67a4522/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:19.408437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.411477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.411477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.490282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 823/1677] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:19 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:19.494703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.542898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] MFA Rules not processed for user `be26427323464b0fbbfd40e7d67a4522`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:19.551827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:19.710975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a80b-9f37-45b8-9959-5d1b0b28bdbb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.732332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.732964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.733240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.733632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.734151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:19.736144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:19.793045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 855/1678] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:19.805882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.812818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:19.833494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:19.869305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 824/1679] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:19 2022] POST /identity/v3/users => generated 288 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:19.872174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.891597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:19.897122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d42d72-89a9-4bc3-b8eb-bf0196adac76 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.914164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.914783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.915061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.915501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:19.916159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:19.921543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:19.946166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 825/1680] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:19.947119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:19.974235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873ce5-9898-410a-95b7-5a06788f58af admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:19.986340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:19.987424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:19.987899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:19.995799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] PATH_INFO: `/v3/projects/65c34691d54c47f3aed7a04eba9d9775/users/42e5520b5112422abdcf650ee790fe94/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.029503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c34691d54c47f3aed7a04eba9d9775, user_id=42e5520b5112422abdcf650ee790fe94, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:20.033509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6895011-0b52-4952-b819-ed1c53fc2de2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:20.049226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c41301-eb3f-4626-91ab-1253b83bcf5a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:20.072926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:20.072926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.072926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.077508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.106648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 826/1681] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:19 2022] PUT /identity/v3/projects/65c34691d54c47f3aed7a04eba9d9775/users/42e5520b5112422abdcf650ee790fe94/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:20.108092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.108213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.108646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.112794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:20.114418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:20.153967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 856/1682] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:20.156518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:20.178057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] MFA Rules not processed for user `02240dc931e140498d82c6d91efb60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:20.179679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:20.197332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a79e18-e51c-4dd2-98c5-e9d54365b1fc service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:20.375276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 827/1683] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:20.375276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:20.411886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:20.412763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.418759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.418759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.423758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:20.431849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:20.459933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 857/1684] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:20.460921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:20.471910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:20.483896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb59be-c558-4a3f-a0f2-d443893bf2e6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:20.487029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:20.487842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.488185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.488526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.490411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:20.644258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:20.648827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:20.671558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 828/1685] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:20.672409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:20.700876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09321e5e-8523-4c4a-a2d0-c117b3665827 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:20.707130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:20.708155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.708582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.709199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] PATH_INFO: `/v3/projects/65c34691d54c47f3aed7a04eba9d9775/users/42e5520b5112422abdcf650ee790fe94/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.714370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c34691d54c47f3aed7a04eba9d9775, user_id=42e5520b5112422abdcf650ee790fe94, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:20.720995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240dc9f-ccd3-425f-9945-8d3a4c7d6864 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:20.754099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 829/1686] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:20 2022] PUT /identity/v3/projects/65c34691d54c47f3aed7a04eba9d9775/users/42e5520b5112422abdcf650ee790fe94/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:20.765702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:20.790156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:20.791524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:20.794806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:20.816070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4a469-734d-4f66-b117-60388c3a5898 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:20.898448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] MFA Rules not processed for user `42e5520b5112422abdcf650ee790fe94`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:20.901631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:20.978407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:21.055520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:21.062192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:21.062192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:21.062192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:21.140810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 830/1687] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:21.167401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:21.169525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:21.214434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:21.220535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158a7f9-c350-4eb2-964b-177f50552e55 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:21.235409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 858/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:21.304244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b928b-e8fd-464a-baa3-77a522bda0b8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:21.308246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:21.308531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:21.308803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:21.318593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:21.320627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:21.321182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:21.322081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:21.347053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:21.349395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca6464-ddb6-46cd-92c7-2a2736e577d8 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:21.373971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] MFA Rules not processed for user `f6d6f82f9aa6447591cdd55d2a040ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:21.374369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 831/1689] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:21.375943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:21.589626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 859/1690] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:21.591026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe16a-8802-4292-8d63-0e0a8f95c00f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:21.591947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe16a-8802-4292-8d63-0e0a8f95c00f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:21.592221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe16a-8802-4292-8d63-0e0a8f95c00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:21.634101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-137fe16a-8802-4292-8d63-0e0a8f95c00f None None] MFA Rules not processed for user `f6d6f82f9aa6447591cdd55d2a040ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:21.634416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-137fe16a-8802-4292-8d63-0e0a8f95c00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:21.650392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 832/1691] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:21.666056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:21.667670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:21.727818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2ecbf-b2e9-4054-99af-fb1f5f441d49 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:21.747833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:21.748743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:21.748743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:21.748743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:21.780041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:21.781952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4420c-ca92-4769-b0d0-b7c35cd5e119 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:21.816366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 860/1692] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:21.884208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:22.904974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:22.905950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:22.906392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:22.908380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 833/1693] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 09:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:22.917120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:22.918085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:22.919135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:23.018862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:23.021629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:23.086273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:23.116793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:23.119238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:23.128165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 861/1694] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:23.158993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b3d0c-1f5b-4457-8ab7-7aa0a635985e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:23.183786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:23.186755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:23.188107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:23.188658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:23.211184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:23.223186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2a067-649f-407d-a8e0-d6b39aaf66e9 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:23.242217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 834/1695] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:23.285891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:23.504846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b9838-7c06-464e-b5d4-9d86875504cb None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:23.508666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:23.510084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:23.511670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:23.512045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] PATH_INFO: `/v3/users/7211fb1a5e0e488086e5221582578c0e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:23.520415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7211fb1a5e0e488086e5221582578c0e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:23.525711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:23.813719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7211fb1a5e0e488086e5221582578c0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:23.855209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7211fb1a5e0e488086e5221582578c0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:23.879444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7211fb1a5e0e488086e5221582578c0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:23.880285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7211fb1a5e0e488086e5221582578c0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:23.882838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7211fb1a5e0e488086e5221582578c0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:23.985264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 862/1696] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:23 2022] DELETE /identity/v3/users/7211fb1a5e0e488086e5221582578c0e => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:23.990228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:23.990681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:23.991065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:24.038177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:24.042775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:24.276482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 835/1697] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:24.348704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:24.349240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:24.562097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50967757-46a2-4d15-b79a-9dc5f01e37cf admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:24.573570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:24.575425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:24.575425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:24.579563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:24.619722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:24.623978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:24.646764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 863/1698] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:25.080136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:25.081905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:25.134674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:25.136136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:25.186752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275560b2-7889-42ba-828d-4be3f3764704 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:25.242555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:25.242555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:25.242555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:25.242555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] PATH_INFO: `/v3/users/fc68ff87997c44e695ef3204f1d57471` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:25.242918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc68ff87997c44e695ef3204f1d57471)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:25.242918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:25.252369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:25.305008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97688184-62d2-4c21-a0f4-6a02eb1ebd70 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:25.315686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:25.315686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:25.315686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:25.315686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:25.318117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:25.335957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:25.344648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ad9a0-1066-4424-bc3f-d8a23e67268b service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:25.370560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 836/1699] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:25.540880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc68ff87997c44e695ef3204f1d57471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:25.594789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc68ff87997c44e695ef3204f1d57471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:25.613964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc68ff87997c44e695ef3204f1d57471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:25.615829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc68ff87997c44e695ef3204f1d57471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:25.617688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc68ff87997c44e695ef3204f1d57471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:25.716711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 864/1700] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:25 2022] DELETE /identity/v3/users/fc68ff87997c44e695ef3204f1d57471 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:25.721464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515a8c4-178a-4347-8880-9bf66a5d3cf0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:25.721753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515a8c4-178a-4347-8880-9bf66a5d3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:25.722029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515a8c4-178a-4347-8880-9bf66a5d3cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:25.769641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8515a8c4-178a-4347-8880-9bf66a5d3cf0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:25.771144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8515a8c4-178a-4347-8880-9bf66a5d3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:25.931088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:25.954250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 837/1701] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:26.035397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:26.041123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:26.119179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:26.120296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:26.122351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:26.200581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa908c-5e2e-4af9-9f5d-902b467322bb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:26.226581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:26.229424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:26.232309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:26.232888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:26.234907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] MFA Rules not processed for user `004ab0c0b138457e9a5e9288d039b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:26.248082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:26.264012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:26.278462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:26.323491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 865/1702] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:26.527641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 838/1703] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:26.545200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:26.546763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:26.789772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48517599-2ae0-4b56-89f8-1bbef5fc92fd service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:26.810070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:26.811359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:26.811854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:26.812326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:26.844534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:26.847231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:26.879364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 866/1704] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:26.923988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:26.924528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:26.957536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688c817-64a7-4de9-af9a-7cd0689ab86a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:26.966487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:26.967706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:26.968212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:26.968740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] PATH_INFO: `/v3/users/4da98704680e4f748e7f582b5d88f68a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:26.970550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da98704680e4f748e7f582b5d88f68a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:26.973198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:27.174193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:27.176057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:27.206747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696d00-20ce-4b40-bcb0-7e202e5d5f03 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:27.212750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da98704680e4f748e7f582b5d88f68a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.235837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:27.238990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:27.240065 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:27.240734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] PATH_INFO: `/v3/projects/d82f55c2986043b08bfe8f562bbb9670` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:27.243213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82f55c2986043b08bfe8f562bbb9670)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:27.251249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da98704680e4f748e7f582b5d88f68a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.251511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:27.277191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:27.387120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da98704680e4f748e7f582b5d88f68a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.387987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da98704680e4f748e7f582b5d88f68a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.388874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f141b5c-636d-488a-b4d9-04b27ec94d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da98704680e4f748e7f582b5d88f68a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.431124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 839/1705] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:26 2022] DELETE /identity/v3/users/4da98704680e4f748e7f582b5d88f68a => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:27.435349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:27.435349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:27.435349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:27.462100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82f55c2986043b08bfe8f562bbb9670', 'request_id': 'req-36fd4b79-3c7b-4a29-a7a1-8623569d686d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.490264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:27.494084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:27.568908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82f55c2986043b08bfe8f562bbb9670', 'request_id': 'req-36fd4b79-3c7b-4a29-a7a1-8623569d686d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.581432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82f55c2986043b08bfe8f562bbb9670', 'request_id': 'req-36fd4b79-3c7b-4a29-a7a1-8623569d686d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.611852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] The token cache is being invalidate because project d82f55c2986043b08bfe8f562bbb9670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:27.613834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.616316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:27.620111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 867/1706] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:27 2022] DELETE /identity/v3/projects/d82f55c2986043b08bfe8f562bbb9670 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:27.625055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:27.628010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:27.688518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 840/1707] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:27.755927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:27.758011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:27.765231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:27.853136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd4b79-3c7b-4a29-a7a1-8623569d686d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:27.884737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:27.885479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:27.885927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:27.886211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] PATH_INFO: `/v3/users/42e5520b5112422abdcf650ee790fe94` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:27.906370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e5520b5112422abdcf650ee790fe94)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:27.906370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:27.971403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0845b3f-4306-4549-994f-c6a73cdeb970 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:27.979893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:27.980521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:27.980803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:27.981667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:27.999568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:28.001105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:28.029846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 841/1708] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:28.221361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e5520b5112422abdcf650ee790fe94'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:28.242075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e5520b5112422abdcf650ee790fe94'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:28.258416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e5520b5112422abdcf650ee790fe94'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:28.259612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e5520b5112422abdcf650ee790fe94'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:28.260514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5771ea62-db52-4fc8-8841-252639a53629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e5520b5112422abdcf650ee790fe94'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:28.358818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 868/1709] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:27 2022] DELETE /identity/v3/users/42e5520b5112422abdcf650ee790fe94 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:28.883716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:28.885497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:28.930385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d112e-01e3-453b-af17-704ac2ebd4c1 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:29.051382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6640a-3019-4d8d-a0c7-0696288d9b6f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.051664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6640a-3019-4d8d-a0c7-0696288d9b6f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.052105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6640a-3019-4d8d-a0c7-0696288d9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.127896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:29.129166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.129460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.129739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] PATH_INFO: `/v3/projects/b85131a2dcda402a8f6aded76993838b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.132179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85131a2dcda402a8f6aded76993838b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:29.134118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:29.161555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-05a6640a-3019-4d8d-a0c7-0696288d9b6f None None] MFA Rules not processed for user `81816508df7c431fa3b4b495d11ac101`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:29.163093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-05a6640a-3019-4d8d-a0c7-0696288d9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.238466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:29.410975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 869/1710] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:29.413821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.414297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.415401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.454409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85131a2dcda402a8f6aded76993838b', 'request_id': 'req-c282e8b8-d8ae-4b55-bc6f-453265584bbe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:29.469384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85131a2dcda402a8f6aded76993838b', 'request_id': 'req-c282e8b8-d8ae-4b55-bc6f-453265584bbe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:29.471362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] MFA Rules not processed for user `81816508df7c431fa3b4b495d11ac101`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:29.473160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.487894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85131a2dcda402a8f6aded76993838b', 'request_id': 'req-c282e8b8-d8ae-4b55-bc6f-453265584bbe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:29.624238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 870/1711] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:29.635893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:29.639219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.643186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] The token cache is being invalidate because project b85131a2dcda402a8f6aded76993838b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:29.649341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:29.649341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c282e8b8-d8ae-4b55-bc6f-453265584bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:29.651417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 842/1712] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:28 2022] DELETE /identity/v3/projects/b85131a2dcda402a8f6aded76993838b => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:29.657211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65647-2d0d-4d1c-8fd4-2351aa010bb6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.658246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65647-2d0d-4d1c-8fd4-2351aa010bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.659593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a65647-2d0d-4d1c-8fd4-2351aa010bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.722786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b4a65647-2d0d-4d1c-8fd4-2351aa010bb6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:29.725104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b4a65647-2d0d-4d1c-8fd4-2351aa010bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.776320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:29.813032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 843/1713] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:29.814710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.815093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.815515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.833642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f485-6f38-4816-8e74-1759970bec56 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:29.840720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] MFA Rules not processed for user `02240dc931e140498d82c6d91efb60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:29.841391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:29.842029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.842313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.842429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.842585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.844413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.859203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:29.860984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793da11-87fb-4911-91bf-89877b665f6f service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:29.872621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 871/1714] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:29.874067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.874315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.874593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.896892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:29.898216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:29.910083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 872/1715] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:29.916992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:29.937481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb967eb9-57c2-47b7-bc06-3bacb9d101b6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:29.956187 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:29.958985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:29.958985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:29.958985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:29.958985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:29.958985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:29.961152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 844/1716] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:29.963619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:29.997090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 873/1717] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:30.002273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.025129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfdb2e-558e-4c62-befb-98b256337735 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.085174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.087660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.089248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.089929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.092239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.099149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51dd5c-acf6-49b8-88cc-31a58f5ac2d9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:30.263388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 845/1718] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:30 2022] POST /identity/v3/projects => generated 363 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:30.273314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.284344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.286152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.286860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.303265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094761-7547-4212-b27f-a77fef4a3bea admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.405904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] MFA Rules not processed for user `3d931a379404433b9c3c690dd9beb83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:30.408515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:30.500351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.501283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.501437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.501828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.502792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.507561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:30.595003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 846/1719] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:30.682020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 874/1720] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:30 2022] POST /identity/v3/users => generated 282 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:30.694515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.720376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24fabc-03e3-4f5b-ace3-fd452a6ea777 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.741243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.741243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.741243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.741243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.741569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.743711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:30.744174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.746072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:30.774432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd0a39-2737-4533-a203-f8639653ec08 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.789743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 847/1721] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:30.793404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.794333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.794831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.795063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.795436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] PATH_INFO: `/v3/projects/7bfd4decd234423a9e98bcc90986c656` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.804714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bfd4decd234423a9e98bcc90986c656)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.807400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:30.814738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731c2d-e96b-467f-8fe1-d70cbf5f449f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.829262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.830089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.830304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.830710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.832157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.835477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:30.868518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 848/1722] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:30.873758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:30.899765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f145de5-0996-4be2-bcb9-09d7575e55d8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:30.931675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:30.931675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:30.931675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:30.931675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] PATH_INFO: `/v3/projects/7e73d4b4da994679934084c351e2f1d8/users/384a0202ebc14765a37355bbeec9314e/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:30.957329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e73d4b4da994679934084c351e2f1d8, user_id=384a0202ebc14765a37355bbeec9314e, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:30.986660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fa944-6d19-4428-adbf-ae3192346ee5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:31.115184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 849/1723] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:30 2022] PUT /identity/v3/projects/7e73d4b4da994679934084c351e2f1d8/users/384a0202ebc14765a37355bbeec9314e/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:31.125331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:31.125617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:31.125899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:31.143325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:31.145126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bfd4decd234423a9e98bcc90986c656', 'request_id': 'req-48374f8b-8712-49a1-991d-5049d600725d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.189204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] MFA Rules not processed for user `384a0202ebc14765a37355bbeec9314e`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:31.190619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:31.242561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bfd4decd234423a9e98bcc90986c656', 'request_id': 'req-48374f8b-8712-49a1-991d-5049d600725d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.251035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bfd4decd234423a9e98bcc90986c656', 'request_id': 'req-48374f8b-8712-49a1-991d-5049d600725d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.300999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] The token cache is being invalidate because project 7bfd4decd234423a9e98bcc90986c656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:31.302488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.308618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.313642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 875/1724] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:30 2022] DELETE /identity/v3/projects/7bfd4decd234423a9e98bcc90986c656 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:31.314742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:31.327757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:31.411204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 850/1725] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:31.414271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:31.422712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:31.502407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48374f8b-8712-49a1-991d-5049d600725d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:31.508085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:31.508740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:31.509021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:31.509291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] PATH_INFO: `/v3/projects/65c34691d54c47f3aed7a04eba9d9775` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:31.515915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c34691d54c47f3aed7a04eba9d9775)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:31.518111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:31.522061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb0e05-74fd-4fbf-b4c3-0da2aa758a76 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:31.548235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:31.552986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:31.552986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:31.552986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] PATH_INFO: `/v3/users/41f934aa0c3d47d9af8b416b9053cf93` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:31.564829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f934aa0c3d47d9af8b416b9053cf93)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:31.564829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:31.582490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:31.695592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:31.718893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c34691d54c47f3aed7a04eba9d9775', 'request_id': 'req-9b559223-752c-44f6-a518-cb0cbcdd2b9a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.888883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c34691d54c47f3aed7a04eba9d9775', 'request_id': 'req-9b559223-752c-44f6-a518-cb0cbcdd2b9a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.894949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c34691d54c47f3aed7a04eba9d9775', 'request_id': 'req-9b559223-752c-44f6-a518-cb0cbcdd2b9a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.937260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] The token cache is being invalidate because project 65c34691d54c47f3aed7a04eba9d9775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:31.937814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.939088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9b559223-752c-44f6-a518-cb0cbcdd2b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:31.957794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 876/1726] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:31 2022] DELETE /identity/v3/projects/65c34691d54c47f3aed7a04eba9d9775 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:31.959401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e643-7f4f-41cc-b4c1-bde376fe2bd2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:31.959691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e643-7f4f-41cc-b4c1-bde376fe2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:31.959968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e643-7f4f-41cc-b4c1-bde376fe2bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:31.995689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f934aa0c3d47d9af8b416b9053cf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:32.046616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f934aa0c3d47d9af8b416b9053cf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:32.088181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-17a4e643-7f4f-41cc-b4c1-bde376fe2bd2 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:32.089707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-17a4e643-7f4f-41cc-b4c1-bde376fe2bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.117598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f934aa0c3d47d9af8b416b9053cf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:32.118243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f934aa0c3d47d9af8b416b9053cf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:32.118874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f934aa0c3d47d9af8b416b9053cf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:32.145629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 851/1727] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:31 2022] DELETE /identity/v3/users/41f934aa0c3d47d9af8b416b9053cf93 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:32.150785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:32.151693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.249946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 877/1728] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:32.251987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.256851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.258648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.264600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 878/1729] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 09:10:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:32.266892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:32.272890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.323606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa62c9-22da-4b79-bac4-ac379ccbc6c0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:32.329396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:32.330295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.331076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.331598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.332636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:32.335080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf11742-b8ab-4830-a023-df667c56a598 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:32.474467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f91a99-b280-44a6-8457-af4cb75b7f9d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:32.483236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:32.483951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.484233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.484511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.486597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.501394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 852/1730] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/projects => generated 365 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:32.501394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.501394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.501945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.530090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:32.531609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.596309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:32.601784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2f0d0-4ccb-433a-8707-36876ae9afcb service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:32.628581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 879/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:32.633132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f6ba-4da3-4ade-995e-647a5e2b5208 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.633431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f6ba-4da3-4ade-995e-647a5e2b5208 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.634667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f6ba-4da3-4ade-995e-647a5e2b5208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.683988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 853/1732] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:32.686738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:32.711659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a884f6ba-4da3-4ade-995e-647a5e2b5208 None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:32.713078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a884f6ba-4da3-4ade-995e-647a5e2b5208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.754424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cf0c9-9d4d-4d32-bb47-f70a3951d9cc None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:32.761771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:32.766115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.766115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.766115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.766115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:32.767692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:32.804966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 880/1733] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:32.807455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117e1cd-fb43-4225-9844-8b0c4b85f931 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.807455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117e1cd-fb43-4225-9844-8b0c4b85f931 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.807681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117e1cd-fb43-4225-9844-8b0c4b85f931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.829788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e117e1cd-fb43-4225-9844-8b0c4b85f931 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:32.831166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e117e1cd-fb43-4225-9844-8b0c4b85f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.840806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 881/1734] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:32.842306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:32.842591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:32.842858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:32.869264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] MFA Rules not processed for user `004ab0c0b138457e9a5e9288d039b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:32.870972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:32.901691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:32.926226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 854/1735] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:32.927296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:32.935092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:33.043207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 882/1736] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:33.046158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.072492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67c61a-cb09-4b0b-9a20-1692b1bcacb4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.083925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.086608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.088577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.089360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.090881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:33.094567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.096665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.137785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 883/1737] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:33.140576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.147840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9281ab-f8fd-4d7c-b090-953e0a692f8c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.156092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.157098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.157546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.157980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.180926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b34f4-9bdd-4f53-a351-22dc7953171b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.199463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.201544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.203480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.204486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.206840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.210000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.240808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 884/1738] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:33.240808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.245159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:33.277340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18eb98-73a3-45ab-934c-d1c401bae8b6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.290582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.292606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.297409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.299908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.301674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.303350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.311677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 855/1739] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:10:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:10:33.311677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.341122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1e15-b9fe-4c72-bb89-4e8991fc3aff service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.351849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.356863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.356863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.356863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.358357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.360463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.361918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.374136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035fbf-e6f6-4670-83a5-75b87cb40bb1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.398812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 885/1740] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:33.415488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.466598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebccbc-1f24-4f3b-873e-0a98ea05552c service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:33.495186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:33.498234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.498234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.498234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] PATH_INFO: `/v3/projects/329ece19b7994c009de17e6529d4c5d9/users/e35858ca7c5e48e4a7c741c38044ab72/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.516162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=329ece19b7994c009de17e6529d4c5d9, user_id=e35858ca7c5e48e4a7c741c38044ab72, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:33.521067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:33.669036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 856/1741] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:33 2022] POST /identity/v3/projects => generated 367 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:33.674681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:33.679774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:33.679962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:33.704107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 886/1742] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:33 2022] PUT /identity/v3/projects/329ece19b7994c009de17e6529d4c5d9/users/e35858ca7c5e48e4a7c741c38044ab72/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:33.705205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:33.710201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:33.742614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:33.744951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:33.901677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 857/1743] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Sep 1 09:10:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:33.908230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:34.008833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff0fa0-1480-47a6-bcdf-c2a58d78fdef admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:34.023898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:34.026858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:34.026858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:34.026858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] PATH_INFO: `/v3/projects/cb2528524b124fafb4af2344a804ec5b` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:34.050948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2528524b124fafb4af2344a804ec5b)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:34.054829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:34.108259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:34.129001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce2782-6be3-49b6-a9b4-26fffbe75acd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:34.148007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:34.148754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:34.149045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:34.149377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:34.172407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:34.180708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:34.337122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2528524b124fafb4af2344a804ec5b', 'request_id': 'req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:34.375350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 858/1744] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:33 2022] POST /identity/v3/users => generated 284 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:34.376486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:34.406337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae698-8eeb-4780-9a76-f7865e0da3da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:34.513381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2528524b124fafb4af2344a804ec5b', 'request_id': 'req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:34.523789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2528524b124fafb4af2344a804ec5b', 'request_id': 'req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:34.571411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:34.571411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:34.571411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:34.571411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:34.571798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:34.571798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:34.576688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] The token cache is being invalidate because project cb2528524b124fafb4af2344a804ec5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:34.576988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:34.593127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-49aae1b7-04a7-4ffa-9b2e-99af18b9de1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:34.601122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 887/1745] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:33 2022] DELETE /identity/v3/projects/cb2528524b124fafb4af2344a804ec5b => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:34.605014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:34.609974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:34.610392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:34.624816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 859/1746] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:34.626985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:34.631684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:34.729006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] MFA Rules not processed for user `81816508df7c431fa3b4b495d11ac101`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:34.730285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:34.893928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1031-4e13-4148-a643-e9fe4c49b69c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:34.910120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:34.910759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:34.911072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:34.917916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:34.917916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:34.937998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:34.941851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:34.957269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 888/1747] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:34.959800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:34.973464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.001594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 860/1748] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:35.004345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:35.010022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.022104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e7b49-77dd-4dfe-bb03-a6a064358c6f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:35.029722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:35.030921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.031410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.031894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.032798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:35.036052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:35.063806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 889/1749] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:35.070808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9b7b3-cef1-46d5-9fe7-7a65d805d233 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:35.076523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:35.079714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.092417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:35.093343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.093862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.094360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] PATH_INFO: `/v3/users/be26427323464b0fbbfd40e7d67a4522` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.096187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be26427323464b0fbbfd40e7d67a4522)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:35.103307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:35.111419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3de41-3056-495c-944f-b5fabb1f87b7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:35.126808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:35.127550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.127826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.128472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] PATH_INFO: `/v3/projects/329ece19b7994c009de17e6529d4c5d9/users/e35858ca7c5e48e4a7c741c38044ab72/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.134628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329ece19b7994c009de17e6529d4c5d9, user_id=e35858ca7c5e48e4a7c741c38044ab72, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:35.141767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:35.181572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 890/1750] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:35 2022] PUT /identity/v3/projects/329ece19b7994c009de17e6529d4c5d9/users/e35858ca7c5e48e4a7c741c38044ab72/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:35.183783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:35.399150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3b2f3-07dc-4436-90b9-4433cd0697d7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:35.402406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be26427323464b0fbbfd40e7d67a4522'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:35.443627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be26427323464b0fbbfd40e7d67a4522'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:35.460461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:35.460461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.460461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.470085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.470928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:35.489940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9acc171-a808-4dac-abd2-e481f765268a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:35.515761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 891/1751] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:35.517294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.517623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.517916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.574044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:35.576981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be26427323464b0fbbfd40e7d67a4522'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:35.581416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be26427323464b0fbbfd40e7d67a4522'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:35.581416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be26427323464b0fbbfd40e7d67a4522'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:35.607386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 861/1752] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:35 2022] DELETE /identity/v3/users/be26427323464b0fbbfd40e7d67a4522 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:35.615029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:35.615029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.645458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] MFA Rules not processed for user `e35858ca7c5e48e4a7c741c38044ab72`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:35.646982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.794576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453874da-41e2-4049-ade0-eb6e9e8714d4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:35.815636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:35.816551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.817023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.817402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] PATH_INFO: `/v3/projects/bbb0bece45d64283a23b1b4d2c0e2cd8/users/401762e538754dd0aefb648b8c47ea99/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:35.851957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb0bece45d64283a23b1b4d2c0e2cd8, user_id=401762e538754dd0aefb648b8c47ea99, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:35.862531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861e4d6-6512-4a8c-909c-a8e99f403737 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:35.900007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 892/1753] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:35.917379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:35.928782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:35.985287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 862/1754] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:35 2022] PUT /identity/v3/projects/bbb0bece45d64283a23b1b4d2c0e2cd8/users/401762e538754dd0aefb648b8c47ea99/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:35.987319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:35.987839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:35.988338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:36.070733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] MFA Rules not processed for user `401762e538754dd0aefb648b8c47ea99`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:36.072300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:36.181413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27b320-ba35-4756-ac14-96cc8724d059 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:36.276809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 863/1755] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:36.282061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:36.380932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:36.381574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:36.381857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:36.382133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:36.522567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926acbf3-5587-4c56-8dab-3b278d2cd9a2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:36.538889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:36.539638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:36.539958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:36.540205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:36.543200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:36.548507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:36.598099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:36.604341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:36.617871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 893/1756] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:36.690633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:36.789865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 864/1757] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:36 2022] POST /identity/v3/projects => generated 367 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:36.791171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:36.809514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd90ee0-6817-4e82-bbc8-23cb7747c171 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:36.951636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:36.990374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:36.990561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:36.990561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:36.990561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:36.991094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:36.991945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:36.995803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.066802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282ce62-96ef-4593-9d5f-8f88bce45b22 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:37.072066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:37.073015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.073580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.074148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:37.077187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:37.147457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 894/1758] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:36 2022] POST /identity/v3/users => generated 290 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:37.151408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:37.161433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:37.164508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.171231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2abcd-ea07-4f5f-bd86-4d8811530ace admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:37.190537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 865/1759] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:37.224177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:37.224177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.224177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.224177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:37.224492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:37.224492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.254282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 895/1760] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:37.257262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:37.286713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4b3c-1ac6-46d4-ac2f-9a5df28e8331 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:37.305889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:37.307992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.309234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.312259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] PATH_INFO: `/v3/projects/61113c3824d24d7d936d45386f1f4557/users/be893c3601f64dd887728ee628e90270/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:37.393312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61113c3824d24d7d936d45386f1f4557, user_id=be893c3601f64dd887728ee628e90270, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:37.400884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.598681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 866/1761] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:37 2022] PUT /identity/v3/projects/61113c3824d24d7d936d45386f1f4557/users/be893c3601f64dd887728ee628e90270/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:37.603053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:37.638774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c253ca-f8e5-4ec6-bc3e-25f6357990db admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:37.807025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:37.810909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.814284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.814732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:37.815639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:37.823301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befa583-ce71-464e-86e7-fcf05aa8a8c4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.847575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 896/1762] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:37.849068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:37.869355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa332af3-ebb5-4ca4-a1bb-1031396c959b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:37.910664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:37.910664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.910664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.910664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] PATH_INFO: `/v3/projects/61113c3824d24d7d936d45386f1f4557/users/be893c3601f64dd887728ee628e90270/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:37.922793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61113c3824d24d7d936d45386f1f4557, user_id=be893c3601f64dd887728ee628e90270, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:37.938258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:37.962749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 867/1763] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:37 2022] PUT /identity/v3/projects/61113c3824d24d7d936d45386f1f4557/users/be893c3601f64dd887728ee628e90270/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:37.975226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:37.975509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:37.975858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:38.021958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] MFA Rules not processed for user `be893c3601f64dd887728ee628e90270`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:38.023644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:38.254027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 897/1764] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:38.267768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:38.394740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4b9ed-afbc-46ef-8558-9378e538a9e4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:38.404590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:38.405566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:38.407274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:38.408587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:38.423912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:38.426079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0857bc-9185-4aa7-9c0e-9ae8e3bba72f service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:38.454532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 868/1765] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:38.692978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:38.717648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:38.719188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:38.789463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957793-8c9d-4fa7-96f7-963db5f48b09 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:38.801028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:38.801945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:38.804739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:38.805402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:39.065685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:39.069700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:39.101919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 898/1766] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:39.278793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:39.377518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874609-e849-408d-b413-a7b4f04ea1f9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:39.378052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874609-e849-408d-b413-a7b4f04ea1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:39.384759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874609-e849-408d-b413-a7b4f04ea1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:39.450011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2f874609-e849-408d-b413-a7b4f04ea1f9 None None] MFA Rules not processed for user `3d931a379404433b9c3c690dd9beb83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:39.459067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2f874609-e849-408d-b413-a7b4f04ea1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:39.598525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:39.600018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:39.691495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 869/1767] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:39.832582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428aa3f-43e5-42d3-81ed-2517af572259 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:39.832582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:39.832582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:39.832582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:39.834176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] PATH_INFO: `/v3/projects/c6a405434bb443bc8c8f567d084744a1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:39.834176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a405434bb443bc8c8f567d084744a1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:39.834176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:40.064741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676158cc-0ae4-45b6-ab92-b0e49288d18f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.065024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676158cc-0ae4-45b6-ab92-b0e49288d18f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.065299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676158cc-0ae4-45b6-ab92-b0e49288d18f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.067165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 870/1768] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:40.088046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a405434bb443bc8c8f567d084744a1', 'request_id': 'req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:40.150793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b5bc-c54e-4ca7-9752-76c47906a084 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.151077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b5bc-c54e-4ca7-9752-76c47906a084 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.151534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b5bc-c54e-4ca7-9752-76c47906a084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.201711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9924b5bc-c54e-4ca7-9752-76c47906a084 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:40.201711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9924b5bc-c54e-4ca7-9752-76c47906a084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:40.221521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a405434bb443bc8c8f567d084744a1', 'request_id': 'req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:40.248231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a405434bb443bc8c8f567d084744a1', 'request_id': 'req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:40.261067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] The token cache is being invalidate because project c6a405434bb443bc8c8f567d084744a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:40.261344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:40.262367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9c91fafa-281a-44c8-9fc2-82a9bef3d08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:40.269791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 899/1769] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:39 2022] DELETE /identity/v3/projects/c6a405434bb443bc8c8f567d084744a1 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:40.280769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.281356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.281808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.373067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:40.373067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:40.506049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 871/1770] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:40.576548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 900/1771] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:40.584558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.584843 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.585114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.657324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:40.667662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:40.699297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 872/1772] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:40.704349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:40.724672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6790a0-ef8a-4650-a77d-fa5e8ad7175c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:40.802334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:40.803745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.804136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.804273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.808978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:40.811081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:40.816133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:40.874985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:40.889174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c438bb-dfd4-4d80-a408-5005cdd5a575 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:40.891660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 901/1773] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:40.917932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:40.918570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:40.918849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:40.920071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:40.921009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:40.923624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:41.064820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 873/1774] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:40 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:41.069489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:41.088321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f096a-b365-4f2b-839f-400ae7d3e562 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:41.279580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:41.285171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:41.285664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:41.286096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:41.286974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:41.290179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:41.607821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 902/1775] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:41 2022] POST /identity/v3/users => generated 292 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:41.608535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:41.634784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d63d-96fa-491b-b3f7-5af07368b5ed admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:41.681034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:41.681034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:41.681034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:41.681034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:41.681881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:41.681881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:41.751692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:41.761712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 874/1776] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:41.767188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3d8c-4103-42d4-bf18-b9ca0eb8d6f3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:41.786899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:41.786899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:41.786899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:41.790076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:41.791663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:41.796124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df4cfc-5bb9-4d3c-92f1-b6f8741abfa7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:41.829452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 903/1777] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:41.833813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:41.857518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7e2a-db8b-48f8-a6d3-b291ad3e3e9a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:41.885107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:41.886011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:41.886367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:41.886707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] PATH_INFO: `/v3/projects/32b89163e2d84931a4fff2c1f0e8a981/users/c026d3d179f144668804948e1a75ee1f/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:41.963517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b89163e2d84931a4fff2c1f0e8a981, user_id=c026d3d179f144668804948e1a75ee1f, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:41.963517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:42.108346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 875/1778] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:41 2022] PUT /identity/v3/projects/32b89163e2d84931a4fff2c1f0e8a981/users/c026d3d179f144668804948e1a75ee1f/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:42.118257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:42.118827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:42.119085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:42.187512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] MFA Rules not processed for user `c026d3d179f144668804948e1a75ee1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:42.190611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:42.277334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:42.324505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 904/1779] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:42.336983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:42.338975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:42.389107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:42.390584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:42.630407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a8347-b94a-430b-9994-e7f907695130 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:42.642444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:42.643882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:42.647324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:42.647756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:42.677316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:42.686001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:42.709888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79d80c-d75a-4f10-b4f4-43c9132de1ad None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:42.720903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 876/1780] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:42.727856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:42.729080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:42.729315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:42.732595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:42.752966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:42.757573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ae25-f838-41d8-a5e0-139babda177f service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:42.809610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 905/1781] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:43.014212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:43.564879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:44.636326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:44.637967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:44.814561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277b9a1-71a0-49d0-a69d-912c0a502397 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:44.831638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:44.832545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:44.833084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:44.833536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] PATH_INFO: `/v3/users/384a0202ebc14765a37355bbeec9314e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:44.834918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=384a0202ebc14765a37355bbeec9314e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:44.837041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:45.178833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384a0202ebc14765a37355bbeec9314e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:45.211092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384a0202ebc14765a37355bbeec9314e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:45.324331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384a0202ebc14765a37355bbeec9314e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:45.328901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384a0202ebc14765a37355bbeec9314e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:45.328901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384a0202ebc14765a37355bbeec9314e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:45.349226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 877/1782] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:44 2022] DELETE /identity/v3/users/384a0202ebc14765a37355bbeec9314e => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:45.352454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ee63-ad9f-4714-9ebb-b77222999898 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:45.352454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ee63-ad9f-4714-9ebb-b77222999898 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:45.352454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ee63-ad9f-4714-9ebb-b77222999898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:45.397255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8e59ee63-ad9f-4714-9ebb-b77222999898 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:45.398775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8e59ee63-ad9f-4714-9ebb-b77222999898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:45.647987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 906/1783] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:45.648381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:45.648381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:45.731723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8fb26-44d0-4102-9b36-099426183ada admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:45.731723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:45.731723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:45.731723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:45.732331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:45.760352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:45.761847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488205e-a4a2-46da-939a-c3b5048515c6 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:45.809045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 878/1784] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:46.445491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:46.445491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:46.445491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:46.445491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 907/1785] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:10:46.485254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:46.485542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:46.485827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:46.566665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:46.572130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:46.824736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 879/1786] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:10:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:46.908057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:47.351585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:47.456108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:47.458189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:47.518514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c51f-efc7-466d-b5fd-63c9dcc4eb57 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:47.544734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:47.545359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:47.545638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:47.551344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:47.594271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:47.609277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d76d65-0ca8-4f27-a38c-320607b4b77d service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:47.635068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 908/1787] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:47.736580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:47.738367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:47.786149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94edbc-27aa-4505-96f3-95e843002a75 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:47.803018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:47.804034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:47.804510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:47.804970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] PATH_INFO: `/v3/users/401762e538754dd0aefb648b8c47ea99` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:47.808484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=401762e538754dd0aefb648b8c47ea99)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:47.811717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:48.084968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '401762e538754dd0aefb648b8c47ea99'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:48.127138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '401762e538754dd0aefb648b8c47ea99'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:48.211894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '401762e538754dd0aefb648b8c47ea99'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:48.212975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '401762e538754dd0aefb648b8c47ea99'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:48.214542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '401762e538754dd0aefb648b8c47ea99'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:48.256937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 880/1788] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:47 2022] DELETE /identity/v3/users/401762e538754dd0aefb648b8c47ea99 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:48.261244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:48.261746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:48.262168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:48.301863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:48.310492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:48.456069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 909/1789] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:48.482581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:48.484365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:48.632636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce846a-975c-466e-aa76-0c979749da28 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:48.642956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:48.643747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:48.644136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:48.644514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:48.653655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:48.655239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:48.682222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c490f6-cf11-40bb-ac8b-2be280260845 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:48.689469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:48.698241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:48.703839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:48.706402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:48.707473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:48.708176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] PATH_INFO: `/v3/projects/7e73d4b4da994679934084c351e2f1d8` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:48.714007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e73d4b4da994679934084c351e2f1d8)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:48.718868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:48.738248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 881/1790] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:49.097634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e73d4b4da994679934084c351e2f1d8', 'request_id': 'req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.183977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e73d4b4da994679934084c351e2f1d8', 'request_id': 'req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.195019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e73d4b4da994679934084c351e2f1d8', 'request_id': 'req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.211527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] The token cache is being invalidate because project 7e73d4b4da994679934084c351e2f1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:49.212068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.212936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.215227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 910/1791] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:48 2022] DELETE /identity/v3/projects/7e73d4b4da994679934084c351e2f1d8 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:49.229874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:49.243333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e74433-253d-4604-981f-c3e8b7a09680 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:49.316742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:49.318475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:49.318475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:49.318475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] PATH_INFO: `/v3/users/e35858ca7c5e48e4a7c741c38044ab72` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:49.319842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35858ca7c5e48e4a7c741c38044ab72)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:49.322023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:49.412026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:49.416207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:49.431890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05ed2c-ad7e-45d2-ae0b-4ff577b99fbc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:49.437517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:49.438156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:49.438435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:49.438703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] PATH_INFO: `/v3/projects/bbb0bece45d64283a23b1b4d2c0e2cd8` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:49.439842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb0bece45d64283a23b1b4d2c0e2cd8)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:49.441859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:49.497071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35858ca7c5e48e4a7c741c38044ab72'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.509869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35858ca7c5e48e4a7c741c38044ab72'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.518304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:49.558182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb0bece45d64283a23b1b4d2c0e2cd8', 'request_id': 'req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.563764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35858ca7c5e48e4a7c741c38044ab72'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.564464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35858ca7c5e48e4a7c741c38044ab72'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.565129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35858ca7c5e48e4a7c741c38044ab72'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.579392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 882/1792] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:49 2022] DELETE /identity/v3/users/e35858ca7c5e48e4a7c741c38044ab72 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:49.639318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb0bece45d64283a23b1b4d2c0e2cd8', 'request_id': 'req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.646116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb0bece45d64283a23b1b4d2c0e2cd8', 'request_id': 'req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.657901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] The token cache is being invalidate because project bbb0bece45d64283a23b1b4d2c0e2cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:49.658518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.658796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.660423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 911/1793] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:49 2022] DELETE /identity/v3/projects/bbb0bece45d64283a23b1b4d2c0e2cd8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:49.664683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:49.666189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:49.737303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bdf6f-3742-4925-9eab-6a5f9509c4f2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:49.741908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:49.742529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:49.742806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:49.743079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] PATH_INFO: `/v3/users/be893c3601f64dd887728ee628e90270` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:49.744234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be893c3601f64dd887728ee628e90270)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:49.746154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:49.927635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be893c3601f64dd887728ee628e90270'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:49.941050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be893c3601f64dd887728ee628e90270'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.084158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be893c3601f64dd887728ee628e90270'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.084920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be893c3601f64dd887728ee628e90270'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.085544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19703268-e77a-40d2-9f20-3f7454609d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be893c3601f64dd887728ee628e90270'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.092659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.094467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.105332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 883/1794] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:49 2022] DELETE /identity/v3/users/be893c3601f64dd887728ee628e90270 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:50.218103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe76a4-6fa4-4aea-93eb-8367dd799cb1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.289652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.289925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.290141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.290410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] PATH_INFO: `/v3/projects/329ece19b7994c009de17e6529d4c5d9` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.291648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329ece19b7994c009de17e6529d4c5d9)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.293710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.411698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329ece19b7994c009de17e6529d4c5d9', 'request_id': 'req-4279d946-27a3-49a8-a0ec-7495f50515e6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.484321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329ece19b7994c009de17e6529d4c5d9', 'request_id': 'req-4279d946-27a3-49a8-a0ec-7495f50515e6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.491789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329ece19b7994c009de17e6529d4c5d9', 'request_id': 'req-4279d946-27a3-49a8-a0ec-7495f50515e6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.503674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] The token cache is being invalidate because project 329ece19b7994c009de17e6529d4c5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:50.503972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.507029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4279d946-27a3-49a8-a0ec-7495f50515e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.507029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 912/1795] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:50 2022] DELETE /identity/v3/projects/329ece19b7994c009de17e6529d4c5d9 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:50.511373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.511851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.512014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.533205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:50.534483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.593762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 884/1796] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:50.601556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.601843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.604762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.606262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.607813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.619519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dedbd-4ac4-4f16-9061-375d3fa590f9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.626132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.626132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.626132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.626132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] PATH_INFO: `/v3/projects/61113c3824d24d7d936d45386f1f4557` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.627472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61113c3824d24d7d936d45386f1f4557)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.629448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.642079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:50.643445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.651743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 913/1797] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:50.656846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.666284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d849f1-1e93-4b6d-a189-d42a09ee34b2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.669085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:50.672811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.673406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.673712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.673978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.674608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:50.674904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.676536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.688583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 914/1798] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:10:50.692772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.702973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fae73d-6df9-4c23-a4d0-40d08a900117 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.707668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.708370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.708651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.708916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.709535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.711445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416e4c1-9cac-4c84-8d65-e3b3f768b4eb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.732096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61113c3824d24d7d936d45386f1f4557', 'request_id': 'req-eed028e0-55d4-4c9e-a97a-08b925f31566'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.737951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61113c3824d24d7d936d45386f1f4557', 'request_id': 'req-eed028e0-55d4-4c9e-a97a-08b925f31566'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.745538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61113c3824d24d7d936d45386f1f4557', 'request_id': 'req-eed028e0-55d4-4c9e-a97a-08b925f31566'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.804124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] The token cache is being invalidate because project 61113c3824d24d7d936d45386f1f4557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:50.804506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.805089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:50.806588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 885/1799] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:50 2022] DELETE /identity/v3/projects/61113c3824d24d7d936d45386f1f4557 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:50.806730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 915/1800] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:50.812134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.818868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88619-558b-4656-82df-828e016cdbde None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.818868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88619-558b-4656-82df-828e016cdbde None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.818868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c88619-558b-4656-82df-828e016cdbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.819136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.832642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-06c88619-558b-4656-82df-828e016cdbde None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:50.833835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-06c88619-558b-4656-82df-828e016cdbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.878259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed028e0-55d4-4c9e-a97a-08b925f31566 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.898644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 916/1801] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:50.908114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.908684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.909051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.930609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:50.931934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:50.941040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.942471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 917/1802] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:50.942936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.943480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.943911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.944709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.946248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.946996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.958275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c187f-0e66-4a58-a600-c4e942fee7e9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.963928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.964539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.964815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.965084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:50.965724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:50.966025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:50.968434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:50.979400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:50.980144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 918/1803] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:50.984543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:50.994129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afdefe-0691-4392-ba7d-25a42487d5e9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:50.998822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:50.999440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:50.999734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:50.999992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.000616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.002505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.034608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 886/1804] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:51.040339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.051504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298d40f-b067-495a-ba6d-3c689807d00a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.056355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.057132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.057409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.057681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.058376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.060094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.070178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 887/1805] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:51.074213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.084788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544c3ee-be96-4868-a4e7-c4eee3e94fca admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.089520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.090123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.090400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.090669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.091444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.093598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.104187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 888/1806] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:51.108319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.117971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04691bc7-bb5f-4b94-adbb-8ec2e9287064 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.122706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.123400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.127635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.127635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] PATH_INFO: `/v3/projects/12446417e3af4d24839a9fb1ad134bf2/users/78b555962ce94c7eafe459816c5df495/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.136163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 919/1807] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:50 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:51.137228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.140557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12446417e3af4d24839a9fb1ad134bf2, user_id=78b555962ce94c7eafe459816c5df495, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.143399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88874f70-801b-4459-8470-1f10cdef86f3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.202124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03833b88-c1c6-4cbe-b4bb-35aebe874b80 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.207266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.207922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.208345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.208568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.209455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.211725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.229311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 889/1808] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:51 2022] PUT /identity/v3/projects/12446417e3af4d24839a9fb1ad134bf2/users/78b555962ce94c7eafe459816c5df495/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:51.242775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.242775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.242775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.273032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] MFA Rules not processed for user `78b555962ce94c7eafe459816c5df495`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:51.274325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:51.335594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 920/1809] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:51.339474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.339619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 890/1810] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:51.343827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.350044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c58e7-ff3a-4d14-8030-8a75c5b84dde admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.416342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78deb63-c4d8-4774-81cd-3121b617c055 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.424750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.425083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.425804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.425961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.426112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.426357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.426473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.426739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.427026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.427537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.428768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.429759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.440388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 921/1811] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:51.444009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.455699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba20ad9-9b3f-45bf-b491-07394609447a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.460692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.461415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.461749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.462026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.462657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.464294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.480147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 922/1812] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:51.484417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.496922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57992494-aa1d-42da-8ec6-197e2d9f8a9b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.500245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.501014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.501389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.501757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] PATH_INFO: `/v3/projects/ddaea8c2ef1c4a90a4a6587ded4a1541/users/c2e442e804074d14a67a32de45e99d37/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.518953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddaea8c2ef1c4a90a4a6587ded4a1541, user_id=c2e442e804074d14a67a32de45e99d37, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.521727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05be6d-be5d-43aa-ad58-4ad7a5bd5ed8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.526686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 891/1813] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:51.531162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.597091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 923/1814] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:51 2022] PUT /identity/v3/projects/ddaea8c2ef1c4a90a4a6587ded4a1541/users/c2e442e804074d14a67a32de45e99d37/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:51.604869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ddf6e-61f9-46a5-8866-d2b5009084d2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.607971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.612067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.612067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.639266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] MFA Rules not processed for user `c2e442e804074d14a67a32de45e99d37`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:51.646362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:51.670966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.674988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.674988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.674988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.674988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.674988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.712921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 924/1815] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:51.717782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.728741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14555d0-c1b4-4d7c-b46c-4b7eec3cf0f3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.734009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.734620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.734893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.735194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.735830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.737736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.821787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 892/1816] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/users => generated 295 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:51.825662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.835655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6f4ba-e881-497b-97c4-9bcdef5920b9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.840245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.840833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.841113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.841387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.842370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.844014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.854063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 893/1817] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:51.858870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.868125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a97e14-3456-4ea6-8034-6453f7412862 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.873255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.873959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.874245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.874511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] PATH_INFO: `/v3/projects/9a34f920fc8d41e095f7234ecbb3290f/users/4de3ee5fa37442c9a8d16c84846c504a/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.888842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a34f920fc8d41e095f7234ecbb3290f, user_id=4de3ee5fa37442c9a8d16c84846c504a, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.894745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 925/1818] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:51.894900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.895718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:51.957655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8304bb-9414-4036-9ed3-c8fcfe0b2d67 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:51.962194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:51.962786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:51.963060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:51.963358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:51.963970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:51.965887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:51.981497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 894/1819] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:51 2022] PUT /identity/v3/projects/9a34f920fc8d41e095f7234ecbb3290f/users/4de3ee5fa37442c9a8d16c84846c504a/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:51.990591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.046360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1445f-b2ba-4a4f-b98b-b9b6f62e4cd7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.051130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.051761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.052129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.052405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.053040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.054681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.067657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 895/1820] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:52.069254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.073910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 926/1821] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:51 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:52.080377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c72fb-54dd-415c-9da5-ee4efcf0080b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.081919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.086015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.087194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.087194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.087387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] PATH_INFO: `/v3/projects/9a34f920fc8d41e095f7234ecbb3290f/users/4de3ee5fa37442c9a8d16c84846c504a/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.090326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a34f920fc8d41e095f7234ecbb3290f, user_id=4de3ee5fa37442c9a8d16c84846c504a, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.093086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825477d-734e-43b0-acdc-f6ecd2389d0b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.094629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29f881-d0c8-41b2-8360-d918c22f23c5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.098559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.099330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.099775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.100146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.100896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.102615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.103830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 896/1822] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:52 2022] PUT /identity/v3/projects/9a34f920fc8d41e095f7234ecbb3290f/users/4de3ee5fa37442c9a8d16c84846c504a/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:52.114108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:52.117915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.117915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.117915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.124132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 927/1823] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:52.128295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.146882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] MFA Rules not processed for user `4de3ee5fa37442c9a8d16c84846c504a`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:52.148173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:52.151978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc824-4c57-4f9d-b40e-809e50792b6b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.235083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 897/1824] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:52.245276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.246772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:52.251521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.252368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.253331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.255653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] PATH_INFO: `/v3/projects/46c16157829445b889726c0d07eaa360/users/6d01855efdd8477bac5b0c6709e34dc9/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.279544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c16157829445b889726c0d07eaa360, user_id=6d01855efdd8477bac5b0c6709e34dc9, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.280051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:52.282389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.310322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214570a-b337-4a4a-9431-2eb479864987 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.313836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.314442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.314715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.318070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.324092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.325517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.335408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 898/1825] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:52.398602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 928/1826] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:52 2022] PUT /identity/v3/projects/46c16157829445b889726c0d07eaa360/users/6d01855efdd8477bac5b0c6709e34dc9/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:52.402326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.414531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac87d-c339-4013-b0dc-a6872a90179e service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.461861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:52.474112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.474785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.475199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.475442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.476195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.477828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105db14-961f-4d78-99c6-03a38c9da776 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.487971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 899/1827] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:52.492193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.502909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9cb487-c7eb-47d4-88a6-4d57c4816ff9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.507685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.508343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.508625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.508900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] PATH_INFO: `/v3/projects/46c16157829445b889726c0d07eaa360/users/6d01855efdd8477bac5b0c6709e34dc9/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.510831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c16157829445b889726c0d07eaa360, user_id=6d01855efdd8477bac5b0c6709e34dc9, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.514061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.524493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 929/1828] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:52 2022] PUT /identity/v3/projects/46c16157829445b889726c0d07eaa360/users/6d01855efdd8477bac5b0c6709e34dc9/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:52.538140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.538140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.538140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.576341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] MFA Rules not processed for user `6d01855efdd8477bac5b0c6709e34dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:52.576341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:52.668638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 900/1829] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:52.677061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:52.743907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132affc6-af57-46a0-b354-138547fd4c05 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:52.747740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:52.748491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:52.748797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:52.749056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:52.805425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:52.806868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:52.815848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 930/1830] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:10:53.540577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:53.642259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e18b8-f0b9-4f80-bc31-ae8a09ba1b94 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:53.647276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4de3ee5fa37442c9a8d16c84846c504a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a34f920fc8d41e095f7234ecbb3290f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:53.648076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:53.648384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:53.648654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:53.649629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:53.649951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:53.651823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:53.668678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 901/1831] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:53.673862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:53.686475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea0b2a-ffac-41dc-8e94-5032e0dd301f service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:53.691272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4de3ee5fa37442c9a8d16c84846c504a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a34f920fc8d41e095f7234ecbb3290f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:53.691827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:53.692125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:53.692403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:53.693077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:53.694992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:53.829229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 931/1832] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:53 2022] POST /identity/v3/projects => generated 355 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:53.833439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:53.845684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733de4-4ef3-4ee6-a734-93c13736d97d tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:53.964898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4de3ee5fa37442c9a8d16c84846c504a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a34f920fc8d41e095f7234ecbb3290f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:53.964898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:53.964898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:53.965163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:53.966069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:53.966915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:53.969626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:54.005627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 902/1833] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:10:54.058832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:54.069682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80da238-d02b-44e5-97b0-7820c29fb282 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:54.074441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4de3ee5fa37442c9a8d16c84846c504a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a34f920fc8d41e095f7234ecbb3290f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:54.075092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:54.075408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:54.075725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] PATH_INFO: `/v3/projects/4ae3294a289c4e2b940eb5bed09d1f29` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:54.077028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4ae3294a289c4e2b940eb5bed09d1f29)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:54.079050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:54.245350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae3294a289c4e2b940eb5bed09d1f29', 'request_id': 'req-45eecea0-82be-4617-81bf-25b683ffe6ec'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:54.316551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae3294a289c4e2b940eb5bed09d1f29', 'request_id': 'req-45eecea0-82be-4617-81bf-25b683ffe6ec'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:54.323131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae3294a289c4e2b940eb5bed09d1f29', 'request_id': 'req-45eecea0-82be-4617-81bf-25b683ffe6ec'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:54.333788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] The token cache is being invalidate because project 4ae3294a289c4e2b940eb5bed09d1f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:54.334071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:54.334726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-45eecea0-82be-4617-81bf-25b683ffe6ec tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:54.336244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 932/1834] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:54 2022] DELETE /identity/v3/projects/4ae3294a289c4e2b940eb5bed09d1f29 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:54.616268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:54.924196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:54.925987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:54.987713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:54.991660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634412a4-d66d-44b3-8c97-926d987d5bb0 tempest-VolumeQuotaClassesTest-71712074 tempest-VolumeQuotaClassesTest-71712074-project-admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:55.050844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:55.051726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:55.052009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:55.052402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] PATH_INFO: `/v3/users/78b555962ce94c7eafe459816c5df495` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:55.053715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b555962ce94c7eafe459816c5df495)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:55.055818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:55.261832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b555962ce94c7eafe459816c5df495'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:55.274066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b555962ce94c7eafe459816c5df495'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:55.340930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b555962ce94c7eafe459816c5df495'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:55.342181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b555962ce94c7eafe459816c5df495'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:55.342805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b555962ce94c7eafe459816c5df495'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:55.355160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 903/1835] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:54 2022] DELETE /identity/v3/users/78b555962ce94c7eafe459816c5df495 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:55.359628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:55.359914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:55.360215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:55.379809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:55.381003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:55.446687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 933/1836] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:55.457123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:55.458650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:55.518245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2ed5a-ff8b-471c-bc7a-93c13e14d4a8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:55.522072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:55.522767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:55.522960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:55.523271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:55.537797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:55.540884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:55.566807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 904/1837] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:56.235919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:56.365504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:56.713331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:56.716158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:56.731474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d7c6-0d4d-4956-a1f5-5d7747855ee2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:56.739027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:56.745091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:56.745561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:56.746024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] PATH_INFO: `/v3/projects/12446417e3af4d24839a9fb1ad134bf2` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:56.748700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12446417e3af4d24839a9fb1ad134bf2)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:56.751193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:56.956416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12446417e3af4d24839a9fb1ad134bf2', 'request_id': 'req-f4cd9244-f70e-4909-8d24-939aab29e34b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.035564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12446417e3af4d24839a9fb1ad134bf2', 'request_id': 'req-f4cd9244-f70e-4909-8d24-939aab29e34b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.044307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12446417e3af4d24839a9fb1ad134bf2', 'request_id': 'req-f4cd9244-f70e-4909-8d24-939aab29e34b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.075252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] The token cache is being invalidate because project 12446417e3af4d24839a9fb1ad134bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:57.075557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.076678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.088256 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:57.093985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 934/1838] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:56 2022] DELETE /identity/v3/projects/12446417e3af4d24839a9fb1ad134bf2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:57.101615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1f-aba8-4462-beeb-f17d8daf45f1 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:57.185022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:57.186925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:57.187240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:57.187516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] PATH_INFO: `/v3/users/4de3ee5fa37442c9a8d16c84846c504a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:57.189053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de3ee5fa37442c9a8d16c84846c504a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:57.192702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:57.472036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de3ee5fa37442c9a8d16c84846c504a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.499930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de3ee5fa37442c9a8d16c84846c504a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.519338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:57.561180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:57.566489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de3ee5fa37442c9a8d16c84846c504a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.567560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de3ee5fa37442c9a8d16c84846c504a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.568232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-520da61b-15e4-40cb-866d-a6cd058b87f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de3ee5fa37442c9a8d16c84846c504a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:57.588066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 905/1839] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:57 2022] DELETE /identity/v3/users/4de3ee5fa37442c9a8d16c84846c504a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:58.166404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.167860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.180933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9244-f70e-4909-8d24-939aab29e34b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.240153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.240795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.241071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.241342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] PATH_INFO: `/v3/projects/9a34f920fc8d41e095f7234ecbb3290f` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.242477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a34f920fc8d41e095f7234ecbb3290f)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.244582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.348784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a34f920fc8d41e095f7234ecbb3290f', 'request_id': 'req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.387519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a34f920fc8d41e095f7234ecbb3290f', 'request_id': 'req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.394903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a34f920fc8d41e095f7234ecbb3290f', 'request_id': 'req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.405841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] The token cache is being invalidate because project 9a34f920fc8d41e095f7234ecbb3290f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:58.406132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.406814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff8e1f64-d72a-4bee-9980-11c9c9ef66f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.408382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 935/1840] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:58 2022] DELETE /identity/v3/projects/9a34f920fc8d41e095f7234ecbb3290f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:10:58.413383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.413672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.414251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.434838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:58.436813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.493077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 906/1841] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:58.502550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.502830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.503112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.522455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:58.523692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.531887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 936/1842] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:58.536117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.546922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46097390-db29-4f5b-b6d2-4af526c1c9ac None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.550915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.552139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.552757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.553034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.553306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.553954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:10:58.554254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.556237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.556352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.570310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302019c-4c7d-4837-ab71-f8a87ed7fcb8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.570490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 907/1843] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:10:58.575432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.575432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.575432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.575432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] PATH_INFO: `/v3/users/c2e442e804074d14a67a32de45e99d37` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.576510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e442e804074d14a67a32de45e99d37)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.576892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.578514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.586491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8045d9-6213-4ce1-8ecc-0dd3542bdd38 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.591198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.592055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.592545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.592831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.593460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.595405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.724520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 908/1844] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:58 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:58.725541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.790813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111ca0a-5502-42d4-a229-05788395f553 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.793083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e442e804074d14a67a32de45e99d37'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.796804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.798320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.798320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.798320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.798642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.800617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.806579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e442e804074d14a67a32de45e99d37'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.814535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e442e804074d14a67a32de45e99d37'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.815430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e442e804074d14a67a32de45e99d37'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.816325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ab6fb200-5f11-4906-9f3d-6e1bf21731e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e442e804074d14a67a32de45e99d37'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:58.869976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 909/1845] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:58 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:58.872178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.874429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.881463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:58.887038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8fa6b-fef8-4892-a2c4-3c25f83417ba admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.888050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 937/1846] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:58 2022] DELETE /identity/v3/users/c2e442e804074d14a67a32de45e99d37 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:58.893168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.893457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.893735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.915402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:58.916742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.941105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:58.959513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.960178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.960466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.960740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.961382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:58.963094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:58.975092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 910/1847] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:58.978791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.985715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 938/1848] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:10:58.992544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682e3b-4f01-4ae1-afe2-86528f39834a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:58.995316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:58.996678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:58.997223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:58.997866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:58.998172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:58.998455 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:58.999295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.000894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.013250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 911/1849] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:59.017338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.026851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ff46-ec51-4914-8969-0ec689ce9092 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.033145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.033875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.035574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.035574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] PATH_INFO: `/v3/projects/629dce3a50e447c48b346c8c5d728b7d/users/531f4c54d6f04f9091adbefc1090f31e/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.052322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629dce3a50e447c48b346c8c5d728b7d, user_id=531f4c54d6f04f9091adbefc1090f31e, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.055267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.064760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169522cf-7244-4bb0-9d7f-59d734cc69bd None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.068877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.070049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.070586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.071190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.083335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.084790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03abf7-e8c8-488f-aa67-56d16f3243ad service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.095626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 939/1850] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:10:59.123458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 912/1851] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:59 2022] PUT /identity/v3/projects/629dce3a50e447c48b346c8c5d728b7d/users/531f4c54d6f04f9091adbefc1090f31e/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:59.133774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.134201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.134583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.174432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] MFA Rules not processed for user `531f4c54d6f04f9091adbefc1090f31e`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:10:59.186450 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:59.270636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 940/1852] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:10:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:10:59.280831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.349801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995de2e-1a60-4a38-958c-f0f9ad52ed06 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.355656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.356260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.356537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.356806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.357568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.359535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.461467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 913/1853] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:10:59 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:10:59.465359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.479496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301fe39-6cea-450a-a899-7060c8fa576a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.597497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.598323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.598838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.599680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.600368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.602315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.662975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.667230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:59.678536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413b981-0f00-4a83-88f3-88a39ca3c1de admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.681199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:10:59.685063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 941/1854] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:10:59 2022] POST /identity/v3/users => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:10:59.687774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.688848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.689301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.689582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] PATH_INFO: `/v3/projects/ddaea8c2ef1c4a90a4a6587ded4a1541` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.691439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddaea8c2ef1c4a90a4a6587ded4a1541)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.693626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.701674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.708427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0608-b566-4260-8cd1-603d1da7fcf9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.713929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.714535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.714816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.715075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.715741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.717690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.727806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 942/1855] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:10:59.732105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.742337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f7830-6e89-45f9-8670-77c6b1d04da0 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.746947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.747623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.748447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.748724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] PATH_INFO: `/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f/users/081486b316e64e1ebdc2d0c9d67e0d6c/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.764202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f655acf1a14b8c9fab8aeb92377e0f, user_id=081486b316e64e1ebdc2d0c9d67e0d6c, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.766899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.833953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 943/1856] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:59 2022] PUT /identity/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f/users/081486b316e64e1ebdc2d0c9d67e0d6c/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:59.837921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.841015 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddaea8c2ef1c4a90a4a6587ded4a1541', 'request_id': 'req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:59.847316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddaea8c2ef1c4a90a4a6587ded4a1541', 'request_id': 'req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:59.854519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddaea8c2ef1c4a90a4a6587ded4a1541', 'request_id': 'req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:59.901430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d95d-2c35-4165-8817-2fce24365ad4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:10:59.936634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] The token cache is being invalidate because project ddaea8c2ef1c4a90a4a6587ded4a1541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:10:59.937016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:59.937874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:10:59.939738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 914/1857] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:10:59 2022] DELETE /identity/v3/projects/ddaea8c2ef1c4a90a4a6587ded4a1541 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:10:59.943440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.945034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:59.961043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:10:59.961581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:10:59.961928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:10:59.962193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:10:59.962890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:10:59.964951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:10:59.976087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 944/1858] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:10:59.980191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:10:59.982781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:10:59.997087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0978d-b9a1-4923-bb24-09b31dc41025 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.004392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.005121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.005229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.005817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] PATH_INFO: `/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f/users/081486b316e64e1ebdc2d0c9d67e0d6c/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.008736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f655acf1a14b8c9fab8aeb92377e0f, user_id=081486b316e64e1ebdc2d0c9d67e0d6c, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.012010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c44c-5aff-4a93-b6f7-26e7478f9406 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.027994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:00.028111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef5f55-b3cf-467f-8e64-eeb3ead1f78b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.037128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 945/1859] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:10:59 2022] PUT /identity/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f/users/081486b316e64e1ebdc2d0c9d67e0d6c/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:00.048526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94121-b798-467d-9161-4e122177c82c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.055692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94121-b798-467d-9161-4e122177c82c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.055692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94121-b798-467d-9161-4e122177c82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.080935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-98e94121-b798-467d-9161-4e122177c82c None None] MFA Rules not processed for user `081486b316e64e1ebdc2d0c9d67e0d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:00.084276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-98e94121-b798-467d-9161-4e122177c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.095472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.096161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.096440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.096714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] PATH_INFO: `/v3/users/6d01855efdd8477bac5b0c6709e34dc9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.097880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d01855efdd8477bac5b0c6709e34dc9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.099879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.159046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 946/1860] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:00.169935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94121-b798-467d-9161-4e122177c82c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:00.171437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-98e94121-b798-467d-9161-4e122177c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.227791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94121-b798-467d-9161-4e122177c82c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.231455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.232037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.232334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.232602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.241573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.242991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.252826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 947/1861] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:00.345051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d01855efdd8477bac5b0c6709e34dc9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.357952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d01855efdd8477bac5b0c6709e34dc9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.413118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:00.425677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf4107-69f1-4ff2-933c-f43a0f8487c7 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.430761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.437451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.437451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.437451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] PATH_INFO: `/v3/users/531f4c54d6f04f9091adbefc1090f31e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.437451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=531f4c54d6f04f9091adbefc1090f31e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.437451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.438743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d01855efdd8477bac5b0c6709e34dc9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.439395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d01855efdd8477bac5b0c6709e34dc9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.440007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d01855efdd8477bac5b0c6709e34dc9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.468175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 915/1862] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:10:59 2022] DELETE /identity/v3/users/6d01855efdd8477bac5b0c6709e34dc9 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:00.578888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:00.580368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.643086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531f4c54d6f04f9091adbefc1090f31e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.645801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65bdf5-b246-4d59-af5b-943fd0016d12 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.650451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.651160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.651442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.651712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] PATH_INFO: `/v3/users/c026d3d179f144668804948e1a75ee1f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.652827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c026d3d179f144668804948e1a75ee1f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.655164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.657598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531f4c54d6f04f9091adbefc1090f31e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.660177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:00.667766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531f4c54d6f04f9091adbefc1090f31e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.669451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531f4c54d6f04f9091adbefc1090f31e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.670926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1cd408de-61bb-4eda-9cc0-4a83710150e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531f4c54d6f04f9091adbefc1090f31e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.793941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 948/1863] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:00 2022] DELETE /identity/v3/users/531f4c54d6f04f9091adbefc1090f31e => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:00.800933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.802180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.802180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.821791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:00.822993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.864329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c026d3d179f144668804948e1a75ee1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.877988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c026d3d179f144668804948e1a75ee1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.883134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 949/1864] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:00.892000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:00.893309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.947638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c026d3d179f144668804948e1a75ee1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.948422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c026d3d179f144668804948e1a75ee1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.949063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c026d3d179f144668804948e1a75ee1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:00.957064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea686-9d30-493e-82de-f25653f66ae7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:00.958300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:00.958919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.959253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.959532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:00.962748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.963886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 916/1865] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:00 2022] DELETE /identity/v3/users/c026d3d179f144668804948e1a75ee1f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:00.964650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:00.965940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:00.976970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:00.978448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe574e0-214e-4de6-abbd-c68ac08bae68 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:00.989115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 950/1866] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:00.990333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:00.990676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:00.990953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.010854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:01.012213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:01.035047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747edd79-0244-4ab1-807d-c69201a2c1da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.041343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.042571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.042879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.043172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] PATH_INFO: `/v3/projects/46c16157829445b889726c0d07eaa360` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.044767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c16157829445b889726c0d07eaa360)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.047896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.095671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 951/1867] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:01.108169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.109634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:01.131071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:01.174978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c900b-3fc3-417a-bd8a-e061c81d6a07 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.179670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c16157829445b889726c0d07eaa360', 'request_id': 'req-47e37bee-320c-4655-baf7-5416cd32e4ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.188166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c16157829445b889726c0d07eaa360', 'request_id': 'req-47e37bee-320c-4655-baf7-5416cd32e4ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.196525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c16157829445b889726c0d07eaa360', 'request_id': 'req-47e37bee-320c-4655-baf7-5416cd32e4ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.237643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.238316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.238600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.238868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.286569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] The token cache is being invalidate because project 46c16157829445b889726c0d07eaa360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:01.286924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.287676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-47e37bee-320c-4655-baf7-5416cd32e4ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.291376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 917/1868] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:00 2022] DELETE /identity/v3/projects/46c16157829445b889726c0d07eaa360 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:01.297873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.298229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.298508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.314924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.316548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342929a-7bd0-42a2-94a5-339b37535992 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.321762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:01.322965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:01.327900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 952/1869] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:01.332720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 918/1870] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:01.345463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.345754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.346035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.366181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:01.367445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:01.371597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:01.377794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 953/1871] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:01.382587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.394227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77911239-f423-4742-8f9b-5e6ccbaad389 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.398893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.399516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.399802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.400069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.400708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:01.401005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.402629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.415328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 919/1872] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:11:01.420465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.430390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bcd94-11e6-4dc6-9ff6-4aebc2712294 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.437115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.437732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.438006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.438276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.438910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.440954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.537473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 954/1873] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:01 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:01.541475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.552736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad7b7-558d-4048-b944-12efe0482f73 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.625965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.627731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:01.660959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.661583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.661858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.662122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.665088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.668464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.731967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541b6be-bc7f-4e5f-8d2e-61a26b6933b4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.737684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.738320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.738597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.739412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] PATH_INFO: `/v3/projects/629dce3a50e447c48b346c8c5d728b7d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.740548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=629dce3a50e447c48b346c8c5d728b7d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.742504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.792738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 920/1874] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:01 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:01.793552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.814534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776b4e3-866e-4f8b-9af3-ba7e9b57481e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.821941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.823259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.823392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.823678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.824659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.832532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.861848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 921/1875] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:01.864985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.883990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabef29-6dae-4a46-b22f-db53052d9f3d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.884465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629dce3a50e447c48b346c8c5d728b7d', 'request_id': 'req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.949661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.950344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.950684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.951175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.951946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:01.953668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:01.961184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629dce3a50e447c48b346c8c5d728b7d', 'request_id': 'req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.964119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 922/1876] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:01.968610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.971858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629dce3a50e447c48b346c8c5d728b7d', 'request_id': 'req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.978239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46f10-9013-4c3e-ab0c-1d53609e371f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:01.984141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] The token cache is being invalidate because project 629dce3a50e447c48b346c8c5d728b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:01.984141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.984985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:01.985192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:01.985800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:01.986082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:01.986376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] PATH_INFO: `/v3/projects/5894998d401942d5bca45e9c4dc9d179/users/a4fb7c7662d04dd887aa77ef2c31a8af/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:01.987261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 955/1877] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:01 2022] DELETE /identity/v3/projects/629dce3a50e447c48b346c8c5d728b7d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:01.990867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:01.992327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.002836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5894998d401942d5bca45e9c4dc9d179, user_id=a4fb7c7662d04dd887aa77ef2c31a8af, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.017135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.025182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc07bd-c1d3-4cc0-8be1-6d5b896b52bf admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.025182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.025182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.025182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.025596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] PATH_INFO: `/v3/users/081486b316e64e1ebdc2d0c9d67e0d6c` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.025596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081486b316e64e1ebdc2d0c9d67e0d6c)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.025596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.086030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 923/1878] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:01 2022] PUT /identity/v3/projects/5894998d401942d5bca45e9c4dc9d179/users/a4fb7c7662d04dd887aa77ef2c31a8af/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:02.087095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.088960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.152396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9913b5-cc07-482a-bd49-c8fd8c3cca5b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.157257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.157873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.158177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.158463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] PATH_INFO: `/v3/projects/32b89163e2d84931a4fff2c1f0e8a981` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.159596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b89163e2d84931a4fff2c1f0e8a981)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.161499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.236754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081486b316e64e1ebdc2d0c9d67e0d6c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.249249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081486b316e64e1ebdc2d0c9d67e0d6c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.256050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081486b316e64e1ebdc2d0c9d67e0d6c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.256739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081486b316e64e1ebdc2d0c9d67e0d6c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.257363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4386cc69-3451-4ba4-bb0c-2f91d7c1df83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081486b316e64e1ebdc2d0c9d67e0d6c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.297745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b89163e2d84931a4fff2c1f0e8a981', 'request_id': 'req-86f42fb0-a37c-4774-babe-882150b8528f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.303556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b89163e2d84931a4fff2c1f0e8a981', 'request_id': 'req-86f42fb0-a37c-4774-babe-882150b8528f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.310173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b89163e2d84931a4fff2c1f0e8a981', 'request_id': 'req-86f42fb0-a37c-4774-babe-882150b8528f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.335388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 956/1879] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:01 2022] DELETE /identity/v3/users/081486b316e64e1ebdc2d0c9d67e0d6c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:02.335388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.335388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.335388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.353839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:02.364497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] MFA Rules not processed for user `a4fb7c7662d04dd887aa77ef2c31a8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:02.365708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.406654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] The token cache is being invalidate because project 32b89163e2d84931a4fff2c1f0e8a981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:02.407135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.408162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-86f42fb0-a37c-4774-babe-882150b8528f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:02.411366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 924/1880] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:02 2022] DELETE /identity/v3/projects/32b89163e2d84931a4fff2c1f0e8a981 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:02.416529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.417125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.417858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.449820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 957/1881] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:02.460375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.461813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.471819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:02.473637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.534475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ba632-4bc0-470b-ba25-ae3ac6a08b47 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.543298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.545677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.545921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.546226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.559733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:02.566318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.570524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c18a16-46ac-432b-a923-61705d391552 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.596427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 958/1882] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:02.605790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 925/1883] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:02.614726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.615009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.615304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.655987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:02.657234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.674803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 959/1884] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:02.678799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.691132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b049d-52f7-4d5b-8a39-01f444e70da4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.703848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.704870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.705156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.705427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.706096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:02.706394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.711379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.736449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 926/1885] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:11:02.740931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.753719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b470f-3288-4f77-9ae7-8b5e4a0816c0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.762478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.763097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.763396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.765025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:02.765658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:02.767782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:02.887844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.895002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 960/1886] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:02 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:02.895162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:02.895261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:02.920971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cbecb-7247-4594-8984-1681b79bc8ce admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.993016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6a01-a8ca-403d-91c8-a9f6571bac34 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:02.996337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.997661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.998795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:02.999403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.999529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:02.999696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:02.999983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.000318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] PATH_INFO: `/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.000609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.001500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f655acf1a14b8c9fab8aeb92377e0f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.002539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.004064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.109969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 961/1887] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:02 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:03.111314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.120848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a371-8b9e-410a-960e-ee42ecfc4b99 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.126265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.126854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.127189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.127461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.128094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.129689 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.142910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 962/1888] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:03.151171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.156636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f655acf1a14b8c9fab8aeb92377e0f', 'request_id': 'req-4df6224a-97b1-4cf8-add9-e68fa109a900'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:03.163029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f655acf1a14b8c9fab8aeb92377e0f', 'request_id': 'req-4df6224a-97b1-4cf8-add9-e68fa109a900'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:03.166738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3ad4-3d46-4425-8f96-62a4bfcde19d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.171766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f655acf1a14b8c9fab8aeb92377e0f', 'request_id': 'req-4df6224a-97b1-4cf8-add9-e68fa109a900'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:03.226586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.227312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.227591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.227862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.228495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.230099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.240399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 963/1889] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:03.244565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.254323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4201-a9e4-4dbb-8ea1-1137f66f1b17 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.259045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.259595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.259876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.260152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] PATH_INFO: `/v3/projects/a384039466ac43eca497c1661c56d1e6/users/405913532e6e4c16a80ea7c7a46d8c5a/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.274832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a384039466ac43eca497c1661c56d1e6, user_id=405913532e6e4c16a80ea7c7a46d8c5a, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.277522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b112a71-748a-43ff-a45b-058276c51c77 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.317202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] The token cache is being invalidate because project 29f655acf1a14b8c9fab8aeb92377e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:03.317538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:03.322180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4df6224a-97b1-4cf8-add9-e68fa109a900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:03.322180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 927/1890] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:02 2022] DELETE /identity/v3/projects/29f655acf1a14b8c9fab8aeb92377e0f => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:03.324323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72919a7-1463-4e0f-bb3e-499c44717725 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.324606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72919a7-1463-4e0f-bb3e-499c44717725 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.324886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72919a7-1463-4e0f-bb3e-499c44717725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.344877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a72919a7-1463-4e0f-bb3e-499c44717725 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:03.346088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a72919a7-1463-4e0f-bb3e-499c44717725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:03.348564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 964/1891] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:03 2022] PUT /identity/v3/projects/a384039466ac43eca497c1661c56d1e6/users/405913532e6e4c16a80ea7c7a46d8c5a/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:03.359244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.359529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.359799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.386994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] MFA Rules not processed for user `405913532e6e4c16a80ea7c7a46d8c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:03.388175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:03.394767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:03.408502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 928/1892] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:03.417751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.418027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.418297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.439924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:03.441254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:03.449792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 929/1893] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:03.453609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.460353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 965/1894] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:03.462843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.464463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:03.480688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cc555-48c1-4bb4-87b1-4743e5876a09 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.486626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.487401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.487513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:03.490912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef660ab-318e-4125-8fe0-a3940773484a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.491591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.492794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.495319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.497309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.497983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.498362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.498519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.499531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:03.501339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.501963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.503616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.528871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 930/1895] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:03.533476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.546728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa3e2e-463a-4985-a498-a24c45842d50 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.570516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.574530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.574530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.574530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.574530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.574985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.630221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 966/1896] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:03.634251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.688817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 931/1897] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:03.693010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.727716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430a820-2207-40b8-9e67-a1b8e3c91b2d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.769209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a33c-082f-417b-9725-adc645621acc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.774538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.776824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.777198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.777554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.778389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.781637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.805201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.806571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.806854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.807184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.807830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.810081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.915288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 932/1898] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:03.919292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.930984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d161d-a7a5-46b0-bfdc-6d830243d7a8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.943722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 967/1899] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:03 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:03.947996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:03.952712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.953362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.953635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.953912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.954572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.963684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41fd44-8bf8-42a0-b86a-60f9b65bca4b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.975036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f0f0d-f9ff-478b-881c-0eb8f2bbfda6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:03.981274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:03.982023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:03.983368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:03.983759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:03.992709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:03.995246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:03.999873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 933/1900] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:04.011616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64f71-733b-4764-bb6f-b4c7665018bd None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.011616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64f71-733b-4764-bb6f-b4c7665018bd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.011616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc64f71-733b-4764-bb6f-b4c7665018bd None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.011616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 934/1901] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:11:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:11:04.015764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc64f71-733b-4764-bb6f-b4c7665018bd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.022236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 968/1902] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:04.028271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.044940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc64f71-733b-4764-bb6f-b4c7665018bd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.069624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945b4e4-1bd8-4509-bd0f-7e0a503b8eb8 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.069624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.069624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.069624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.070414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.070414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.070414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846bda3-ab30-4a36-947b-1fe4e9ba3e83 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.070989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.070989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.070989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.071594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.072426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.074862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.080136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 969/1903] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:04.087172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.087172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.087172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.103460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 935/1904] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:04.104681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.118562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:04.126476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bdc35-fb2a-48c3-bf7b-dee03d4827d9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.127464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:04.127567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.127567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.129310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.129310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] PATH_INFO: `/v3/projects/f52aa5afb5694a8f98f36b8118c7e330/users/fe1e4e89fb284fed88f7e5c2558edac1/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.147492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52aa5afb5694a8f98f36b8118c7e330, user_id=fe1e4e89fb284fed88f7e5c2558edac1, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.152151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f7a87-c813-40f9-83c6-0a6f96e7e11e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.202846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 970/1905] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:04.207385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.220905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e848-ff3f-49e3-89b5-d87a89ff6be0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.226576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.227351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.227695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.228152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] PATH_INFO: `/v3/projects/c983615314ee463c8b64ef214855c575/users/e6e5fe95a70d445a8c5a4f526f0a0b00/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.246356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c983615314ee463c8b64ef214855c575, user_id=e6e5fe95a70d445a8c5a4f526f0a0b00, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.249805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c23639-c170-4327-8846-3ce4204eba3b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.343788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 936/1906] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:04 2022] PUT /identity/v3/projects/f52aa5afb5694a8f98f36b8118c7e330/users/fe1e4e89fb284fed88f7e5c2558edac1/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:04.356198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9251c27-f571-4510-94f1-545acad3089f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.356710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9251c27-f571-4510-94f1-545acad3089f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.357178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9251c27-f571-4510-94f1-545acad3089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.393260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 971/1907] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:04 2022] PUT /identity/v3/projects/c983615314ee463c8b64ef214855c575/users/e6e5fe95a70d445a8c5a4f526f0a0b00/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:04.395038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.395615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.398437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.401190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c9251c27-f571-4510-94f1-545acad3089f None None] MFA Rules not processed for user `fe1e4e89fb284fed88f7e5c2558edac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:04.402778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c9251c27-f571-4510-94f1-545acad3089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:04.459197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] MFA Rules not processed for user `e6e5fe95a70d445a8c5a4f526f0a0b00`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:04.461779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:04.565079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 937/1908] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:04.570710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9251c27-f571-4510-94f1-545acad3089f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.612898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 972/1909] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:04.613787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.619090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:04.749611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9251c27-f571-4510-94f1-545acad3089f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.763324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.763973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.764607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.764890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.765648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.773418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.794283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639b6c8-645f-409a-98fd-19d8ce09c064 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.808636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.814426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.814426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.814426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.824541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.826541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.839163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 973/1910] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:04.839946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:04.858241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a98f-6545-4edb-aba6-633c650b2d06 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:04.868350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:04.869288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:04.869733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:04.870620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:04.870741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:04.873676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:04.908584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 938/1911] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/projects => generated 375 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:04.912852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.025136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384b4d2-b467-4ca1-92aa-6d973e26c4d1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.034384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 974/1912] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:05.035482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.036686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.037114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.037504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.038640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.040482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.045538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.118280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b00f5-e981-482a-b928-3329930e80e6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.124507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.125555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.125991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.126427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.127355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.129445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.173723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 939/1913] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:04 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:05.178074 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.190223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f54f5-c184-45be-aeec-cdf22cab2d15 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.196959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.197995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.198482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.198995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.199964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.202533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.215174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 940/1914] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:05.220913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.233132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 975/1915] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:05 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:05.235519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.240310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32667812-d5e3-4f78-95b8-34ac4b3b1e99 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.246696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.247570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.248041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.248970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] PATH_INFO: `/v3/projects/1adccd1f23104511b5c71e7911c74d5e/users/a7f7df4763324e34b5cb1a6840067e87/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.264364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:05.268522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2769635-2ff1-4c79-8858-0dbe53fb7138 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.290177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.291287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.291758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.299268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.299268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.299268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.299545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adccd1f23104511b5c71e7911c74d5e, user_id=a7f7df4763324e34b5cb1a6840067e87, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.301747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.315027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 976/1916] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:05.325381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.346854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843a5da-c40b-4115-a628-13487264f05a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.348459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:05.353609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.355360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.360310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.360310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] PATH_INFO: `/v3/projects/17a0b6ad909e4c2589352a9bc56138d7/users/2fe48f617c99484e82acc85e137d0f89/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.377149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 941/1917] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:05 2022] PUT /identity/v3/projects/1adccd1f23104511b5c71e7911c74d5e/users/a7f7df4763324e34b5cb1a6840067e87/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:05.380713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a0b6ad909e4c2589352a9bc56138d7, user_id=2fe48f617c99484e82acc85e137d0f89, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.383806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.384444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.458460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 977/1918] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:05 2022] PUT /identity/v3/projects/17a0b6ad909e4c2589352a9bc56138d7/users/2fe48f617c99484e82acc85e137d0f89/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:05.460603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.497642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea5060-5d37-4eb9-b977-d9f7898c7c92 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.557572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7830a70-bd1d-48f6-958a-51c54ce1ca41 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.566702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.568312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.568608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.568886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.570054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.573699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.613835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 978/1919] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:05.614818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.627590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624734b-f0a3-467b-b65c-4e932cafaf09 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.633009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.633690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.634251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.634540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] PATH_INFO: `/v3/projects/17a0b6ad909e4c2589352a9bc56138d7/users/2fe48f617c99484e82acc85e137d0f89/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.637555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.637872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a0b6ad909e4c2589352a9bc56138d7, user_id=2fe48f617c99484e82acc85e137d0f89, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.638381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.640058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.640439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.642233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe931f-0ea8-4d3e-a6d1-8e7b767e7f74 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.642608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.646920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.670924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 979/1920] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:05 2022] PUT /identity/v3/projects/17a0b6ad909e4c2589352a9bc56138d7/users/2fe48f617c99484e82acc85e137d0f89/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:05.672764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.673056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.673265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 942/1921] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:05.673369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.674464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.714567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7518a-5a73-43a2-a4fb-61195b2cddc4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:05.717517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] MFA Rules not processed for user `2fe48f617c99484e82acc85e137d0f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:05.718655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:05.886445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:05.887126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.887387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.887716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] PATH_INFO: `/v3/projects/1adccd1f23104511b5c71e7911c74d5e/users/a7f7df4763324e34b5cb1a6840067e87/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.888311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 980/1922] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:05.891671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adccd1f23104511b5c71e7911c74d5e, user_id=a7f7df4763324e34b5cb1a6840067e87, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:05.894178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38330b-2233-4ac3-8d78-9fd7f8eb5efa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:05.903508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:05.907028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:05.908546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 943/1923] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:05 2022] PUT /identity/v3/projects/1adccd1f23104511b5c71e7911c74d5e/users/a7f7df4763324e34b5cb1a6840067e87/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:05.923814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:05.924793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:05.925069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:05.973948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] MFA Rules not processed for user `a7f7df4763324e34b5cb1a6840067e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:05.973948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:06.036966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78903c9a-57f8-42e9-86a1-601fc6cc7623 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:06.159358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 944/1924] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:06.166348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:06.166986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:06.167288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:06.168159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:06.244914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:06.246360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:06.263482 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:06.265460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:06.274603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c873-4cc6-4cbd-a6dc-f36b8eead0a4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:06.281996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:06.282820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:06.283330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:06.283767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:06.339331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 981/1925] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:06.388263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:06.389941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:06.400024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 945/1926] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:06.791612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:06.802537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b69be7-2c8b-46c8-b48f-2614d53e77b1 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:06.807710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:06.808327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:06.808634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:06.808966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:06.832174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:06.880123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:06.900883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:06.900883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:06.909203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 982/1927] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:07.767684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:09.411271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:09.542320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:09.551738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb14e-4e4a-4d6f-85d2-177005aa7f45 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:09.555938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:09.556899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:09.557286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:09.557664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:09.620529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:09.621995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f54ca4-7906-4c33-ad8f-bd32b61d8ef9 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:09.630806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 946/1928] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:09.725214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:10.401285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:10.402723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:10.470401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf862-7083-435b-9ce6-522a6b5ef0b6 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:10.475711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:10.478797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:10.478797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:10.478797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] PATH_INFO: `/v3/users/a4fb7c7662d04dd887aa77ef2c31a8af` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:10.478797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fb7c7662d04dd887aa77ef2c31a8af)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:10.479952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:10.542446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:10.701175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fb7c7662d04dd887aa77ef2c31a8af'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:10.714076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fb7c7662d04dd887aa77ef2c31a8af'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:10.721424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fb7c7662d04dd887aa77ef2c31a8af'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:10.722154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fb7c7662d04dd887aa77ef2c31a8af'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:10.722766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fb7c7662d04dd887aa77ef2c31a8af'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:10.803832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:10.868181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 983/1929] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:10 2022] DELETE /identity/v3/users/a4fb7c7662d04dd887aa77ef2c31a8af => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:10.872778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:10.873178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:10.873564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:10.897285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:10.899856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:10.957421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 947/1930] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:10.967279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:10.968771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:11.026520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d481b2-2fb0-4eda-bbd9-3f6d07b7aa1f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:11.030195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:11.032168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:11.032168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:11.032168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:11.043064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:11.044523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:11.054821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 984/1931] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:11.313225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:11.393781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:11.395580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:11.458435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90170a44-41b1-4e2b-9bab-22fc4ea1cf88 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:11.462201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:11.462822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:11.463203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:11.463680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:11.465531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:11.473253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:11.538761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:11.541060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873ea96-3fed-4a0b-ba40-7664a14397b1 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:11.557016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 948/1932] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:11.720727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:11.722149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:11.750956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0a8d-162b-40d1-bdd4-a93710f44f7b service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:11.751765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:11.752418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:11.752691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:11.753195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] PATH_INFO: `/v3/projects/5894998d401942d5bca45e9c4dc9d179` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:11.754537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5894998d401942d5bca45e9c4dc9d179)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:11.756592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:11.903471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5894998d401942d5bca45e9c4dc9d179', 'request_id': 'req-a7f63dbd-f958-409a-8f3a-3052747a00c7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:11.973186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5894998d401942d5bca45e9c4dc9d179', 'request_id': 'req-a7f63dbd-f958-409a-8f3a-3052747a00c7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:11.981382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5894998d401942d5bca45e9c4dc9d179', 'request_id': 'req-a7f63dbd-f958-409a-8f3a-3052747a00c7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:11.998363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] The token cache is being invalidate because project 5894998d401942d5bca45e9c4dc9d179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:11.999040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:12.000412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a7f63dbd-f958-409a-8f3a-3052747a00c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:12.002159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 985/1933] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:11 2022] DELETE /identity/v3/projects/5894998d401942d5bca45e9c4dc9d179 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:12.007873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.009349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.009860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.040323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:12.041782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:12.051548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:12.127719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 949/1934] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:12.137587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.138432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.138878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.167049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:12.168348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:12.177327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 986/1935] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:12.185476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.196464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042de03-ca61-4145-8bbb-0b56cad75fe2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.201794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.202013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.202288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.203206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.204004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:12.204396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.205971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.220498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 950/1936] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:11:12.227606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.238275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e590e-1c9a-4481-8c7b-f479d523ec23 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.243131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.244087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.244087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.244439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.245117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.247155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.387534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 987/1937] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:12 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:12.388670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.400392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae004e-e9e3-4140-98e1-b271ff4fc1f6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.458408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:12.472565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.473250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.473535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.473814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.474449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.476703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.576508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 951/1938] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:12 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:12.580580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.604868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaf9b3-636e-47aa-9926-276c72ef4ea6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.623154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.624149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.625291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.625645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.627320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.634791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.652067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.655208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 988/1939] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:12.677730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523d4d6-f9c2-4e57-a4b4-4f87577e885e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.694866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.695564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.695836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.696106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.696768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.698391 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e582751-240c-42fc-89c3-b90c4a3d11b7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.721911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 952/1940] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:12.726654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:12.737694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a222b-fd32-42a9-91ed-9e2c7f516820 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:12.742725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:12.743562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.743911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.744252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] PATH_INFO: `/v3/projects/1393fb2e7bd84f70a533e2200226ca99/users/d721b90127ee46d1949bbf412c74f399/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.763941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1393fb2e7bd84f70a533e2200226ca99, user_id=d721b90127ee46d1949bbf412c74f399, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:12.768054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:12.874421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 989/1941] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:12 2022] PUT /identity/v3/projects/1393fb2e7bd84f70a533e2200226ca99/users/d721b90127ee46d1949bbf412c74f399/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:12.884480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:12.884768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:12.885044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:12.914269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] MFA Rules not processed for user `d721b90127ee46d1949bbf412c74f399`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:12.915818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:12.995029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:13.010683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 953/1942] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:13.015821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.092406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3b4b2-39f8-4623-8fc3-1d848c110254 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.097612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.097974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.098259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.098538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.099209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.101160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.167150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 990/1943] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:13 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:13.171602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.182186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d71440-36ef-4871-9c06-56cd44d45a77 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.247790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.248405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.248690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.248955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.250170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.252210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.355707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 954/1944] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:13 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:13.366250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.378145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2dd2-af54-4b87-984a-a56818206779 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.383337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.383937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.387089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.387089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.387089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.387089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.401148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 991/1945] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:13.402643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.412914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0af1-e108-4345-8484-b2317deab96c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.420084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.420993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.421885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.422374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.423405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.425113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3a013-f7b0-4ea9-b39e-b63c792da489 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.441502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 955/1946] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:13.445920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.458286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece0fb8-b703-4ab6-b18a-fad9e326dc82 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.472125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.472836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.473347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.473634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] PATH_INFO: `/v3/projects/7073dafd36b74983b871dbfb32d21376/users/06d112c89331456fbf47073ccc199780/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.495935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7073dafd36b74983b871dbfb32d21376, user_id=06d112c89331456fbf47073ccc199780, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.500466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.588881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 992/1947] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:13 2022] PUT /identity/v3/projects/7073dafd36b74983b871dbfb32d21376/users/06d112c89331456fbf47073ccc199780/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:13.600336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.601246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.601521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.634727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] MFA Rules not processed for user `06d112c89331456fbf47073ccc199780`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:13.635933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:13.705044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 956/1948] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:13.711077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.720553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:13.778867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2053a5-ff55-4bbc-afd5-b3068baf229c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.784368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.790663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.790663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.790663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.790663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.790663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:13.893615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 993/1949] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:13 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:13.901215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:13.912373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9ae51-171b-4bae-bb81-9f67a3883ce8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:13.959769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:13.967964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:13.969125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:13.969524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:13.969769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:13.970421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:13.972439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.088165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 957/1950] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:13 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:14.089335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.100578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb366-082d-4257-8e9e-c1d76e98d22f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.105186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.105782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.106059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.106332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.106976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.108637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.121054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 994/1951] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:14.125398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.135558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87390f7-b818-43a1-8445-3d3b717868ed admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.142564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.143187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.143468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.148984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] PATH_INFO: `/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa/users/ce6cc3a2ea9241009a29ae206564dbdc/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.160592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2767bd7ce94449cc9011a4bdb50b18aa, user_id=ce6cc3a2ea9241009a29ae206564dbdc, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.163432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.189808 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:14.240142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 958/1952] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:14 2022] PUT /identity/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa/users/ce6cc3a2ea9241009a29ae206564dbdc/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:14.244163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.256440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2297e5a-34af-49e3-97ea-6ced65d4f5f2 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.297358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:14.324252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.324889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.325167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.325439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.326075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.327865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9066d-2baa-4b76-b4f8-7b3a15deece4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.338285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 995/1953] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:14.342351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.353237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e897-4ac0-49d6-9c75-9e6902371b8b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.357866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.358512 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.359364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.359864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] PATH_INFO: `/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa/users/ce6cc3a2ea9241009a29ae206564dbdc/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.361995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2767bd7ce94449cc9011a4bdb50b18aa, user_id=ce6cc3a2ea9241009a29ae206564dbdc, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.364736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.375926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 959/1954] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:14 2022] PUT /identity/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa/users/ce6cc3a2ea9241009a29ae206564dbdc/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:14.390671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.391306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.391779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.433185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] MFA Rules not processed for user `ce6cc3a2ea9241009a29ae206564dbdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:14.434667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:14.506662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 996/1955] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:14.519840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.522480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:14.627882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc03312-2bb9-4527-9d9a-2138f0659925 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.633468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.634110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.634383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.634659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.708626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.710511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05abb3-8b7c-4cf6-8211-57f0a076c159 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:14.726601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 960/1956] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:14.776737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:14.778288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:14.823243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:14.881512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba095e-c8ec-4a6d-8ac5-ea7c251742e6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:14.897059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:14.897986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:14.899193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:14.901368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] PATH_INFO: `/v3/users/e6e5fe95a70d445a8c5a4f526f0a0b00` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:14.905481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e5fe95a70d445a8c5a4f526f0a0b00)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:14.908411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:15.052462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:15.162313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e5fe95a70d445a8c5a4f526f0a0b00'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:15.179677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e5fe95a70d445a8c5a4f526f0a0b00'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:15.182712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e5fe95a70d445a8c5a4f526f0a0b00'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:15.183463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e5fe95a70d445a8c5a4f526f0a0b00'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:15.184075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e5fe95a70d445a8c5a4f526f0a0b00'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:15.273295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 997/1957] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:14 2022] DELETE /identity/v3/users/e6e5fe95a70d445a8c5a4f526f0a0b00 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:15.277791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:15.278146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:15.283036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:15.298112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:15.299341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:15.365797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 961/1958] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:15.378302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:15.380704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:15.450389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982e4f-c918-4cc6-9582-83ffaaf23503 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:15.455379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:15.457795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:15.457795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:15.457795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:15.466307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:15.467840 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:15.477559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 998/1959] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:15.738579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:16.042486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:16.044160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:16.059465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e54c7-5148-4538-a577-2f0636bf1b81 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:16.070682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:16.071552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:16.071840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:16.072118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] PATH_INFO: `/v3/projects/c983615314ee463c8b64ef214855c575` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:16.073272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c983615314ee463c8b64ef214855c575)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:16.075359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:16.210868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c983615314ee463c8b64ef214855c575', 'request_id': 'req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.221730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c983615314ee463c8b64ef214855c575', 'request_id': 'req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.232132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c983615314ee463c8b64ef214855c575', 'request_id': 'req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.247223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:16.308426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] The token cache is being invalidate because project c983615314ee463c8b64ef214855c575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:16.308718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.309368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.311738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 962/1960] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:16 2022] DELETE /identity/v3/projects/c983615314ee463c8b64ef214855c575 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:16.316172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:16.328033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669d053-50a4-4eaa-ae79-494806b0438f service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:16.400708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:16.401398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:16.401836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:16.402132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] PATH_INFO: `/v3/users/2fe48f617c99484e82acc85e137d0f89` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:16.403520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe48f617c99484e82acc85e137d0f89)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:16.405481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:16.433981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:16.638080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe48f617c99484e82acc85e137d0f89'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.655035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe48f617c99484e82acc85e137d0f89'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.663836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe48f617c99484e82acc85e137d0f89'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.665107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe48f617c99484e82acc85e137d0f89'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.665788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe48f617c99484e82acc85e137d0f89'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:16.748410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 999/1961] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:16 2022] DELETE /identity/v3/users/2fe48f617c99484e82acc85e137d0f89 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:17.274128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:17.278501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:17.292371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1756b-cc0e-4f42-9b98-4cbe7b590a38 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:17.305383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:17.364744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:17.365801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.366331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.366674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] PATH_INFO: `/v3/projects/17a0b6ad909e4c2589352a9bc56138d7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.368047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a0b6ad909e4c2589352a9bc56138d7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:17.370241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:17.377433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:17.470489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a0b6ad909e4c2589352a9bc56138d7', 'request_id': 'req-0370f502-1f58-4d8c-ae8e-161c36b025d5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.475838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a0b6ad909e4c2589352a9bc56138d7', 'request_id': 'req-0370f502-1f58-4d8c-ae8e-161c36b025d5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.481992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a0b6ad909e4c2589352a9bc56138d7', 'request_id': 'req-0370f502-1f58-4d8c-ae8e-161c36b025d5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.582109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] The token cache is being invalidate because project 17a0b6ad909e4c2589352a9bc56138d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:17.589250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.589250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0370f502-1f58-4d8c-ae8e-161c36b025d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.589435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:17.589527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 963/1962] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:17 2022] DELETE /identity/v3/projects/17a0b6ad909e4c2589352a9bc56138d7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:17.589610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:17.593789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa40b9-da72-4c5f-9b84-59172b2bbf0f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.594325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa40b9-da72-4c5f-9b84-59172b2bbf0f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.594437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa40b9-da72-4c5f-9b84-59172b2bbf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.599477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2dbb-fc74-4a0f-8217-c05d520dea47 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:17.615785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bfaa40b9-da72-4c5f-9b84-59172b2bbf0f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:17.616937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bfaa40b9-da72-4c5f-9b84-59172b2bbf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:17.661657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:17.662291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.662569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.662838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] PATH_INFO: `/v3/users/405913532e6e4c16a80ea7c7a46d8c5a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.664013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405913532e6e4c16a80ea7c7a46d8c5a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:17.666214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:17.682825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 964/1963] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:17.690915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.691281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.691676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.711368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:17.712873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:17.721848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 965/1964] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:17.725911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:17.735389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940daa15-3971-4362-bbc6-88b9e6457ade None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:17.739984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:17.740580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.740859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.741133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.741774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:17.742073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:17.743706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:17.755857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 966/1965] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:17.760121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:17.770607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8567a6-1553-4ffd-a9f4-419c0dd9262b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:17.775316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:17.779568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:17.779568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:17.779568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:17.779568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:17.779568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:17.893242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405913532e6e4c16a80ea7c7a46d8c5a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.905385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405913532e6e4c16a80ea7c7a46d8c5a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.913900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405913532e6e4c16a80ea7c7a46d8c5a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.914871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405913532e6e4c16a80ea7c7a46d8c5a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.915509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-730c2636-5f63-4161-9d98-e90ab3bce729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405913532e6e4c16a80ea7c7a46d8c5a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:17.982979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:17.992120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 967/1966] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:17 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:17.995565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:17.997193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.066832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dbf2d-ca5b-4800-9d60-5d1864825054 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.072012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.072737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.073018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.073871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.074639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.076637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.092267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1000/1967] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:17 2022] DELETE /identity/v3/users/405913532e6e4c16a80ea7c7a46d8c5a => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:18.097227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.097706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.098079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.119413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:18.120731 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.185496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 968/1968] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:17 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:18.186706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.206765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deef1da-1d16-44de-bea8-07870f56d8f1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.213093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.213093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.213093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.213442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.213956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.215750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.216215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1001/1969] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:18.226502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.228597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.237545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 969/1970] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:18.238584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.240915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.299656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:18.304657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16741cd8-4d7f-458a-a0a0-d2eca4f99319 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.315700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.316326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.316600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.316868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.320324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8021c22-6e8a-4880-9df4-25c1806dd3cd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.321649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.326777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.327439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.328117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.328704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.343990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.345826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.360600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 970/1971] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:18.361702 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.363610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:18.376316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a630cd-7776-40ba-af99-ff74cce1593a service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.392638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.394099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.396910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.397312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.399743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.405708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.428184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 971/1972] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:18.433337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.444251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.445817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff9e2-f9ac-40e9-a6f9-17e6b6f380c6 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.467324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1002/1973] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:18.472233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f743ed-3ea7-44ca-868c-1258c4df501d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.477931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.478697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.479076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.479493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] PATH_INFO: `/v3/projects/b9eef1a12519460bbd0e1e7da5b88a42/users/2cccca11a180451c98d45f8dc6adbbf6/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.497821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9eef1a12519460bbd0e1e7da5b88a42, user_id=2cccca11a180451c98d45f8dc6adbbf6, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.501760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.611357 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 972/1974] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:18 2022] PUT /identity/v3/projects/b9eef1a12519460bbd0e1e7da5b88a42/users/2cccca11a180451c98d45f8dc6adbbf6/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:18.622184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.622473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.622751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.653898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] MFA Rules not processed for user `2cccca11a180451c98d45f8dc6adbbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:18.658104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.733359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1003/1975] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:18.741342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.793125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:18.826228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e36d0-e4c0-4799-8b63-bf29bf02d7b2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.840712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.840712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.840712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.840712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.841319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.841319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:18.967130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:18.969216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:18.983955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e82be-410e-4fef-9640-e3f638c33583 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:18.989632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:18.990279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:18.990550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:18.990982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] PATH_INFO: `/v3/projects/a384039466ac43eca497c1661c56d1e6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:18.992316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a384039466ac43eca497c1661c56d1e6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:18.994236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.003864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 973/1976] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:18 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:19.010278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.030075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:19.084118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1976c-5e60-403d-aed9-768c0a47c89e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.089367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.090003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.090379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.090666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.091575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.093791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.151394 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a384039466ac43eca497c1661c56d1e6', 'request_id': 'req-9ed9994b-d7e4-4dd0-bdc5-807daf319988'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.218021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 974/1977] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:19 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:19.222201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.236123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08e97b-d675-466e-89f2-8744d27cc8e8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.245231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a384039466ac43eca497c1661c56d1e6', 'request_id': 'req-9ed9994b-d7e4-4dd0-bdc5-807daf319988'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.248032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a384039466ac43eca497c1661c56d1e6', 'request_id': 'req-9ed9994b-d7e4-4dd0-bdc5-807daf319988'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.261057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] The token cache is being invalidate because project a384039466ac43eca497c1661c56d1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:19.261507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.262195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.264034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1004/1978] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:18 2022] DELETE /identity/v3/projects/a384039466ac43eca497c1661c56d1e6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:19.270514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.272371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:19.340397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.341213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.342675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.342969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.344282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.345981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.350782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9994b-d7e4-4dd0-bdc5-807daf319988 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.356353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.356629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.356906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.357175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] PATH_INFO: `/v3/users/fe1e4e89fb284fed88f7e5c2558edac1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.358505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 975/1979] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:19.359332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1e4e89fb284fed88f7e5c2558edac1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.362204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.362684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.364430 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:19.377134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f41a-0d03-4d7f-9c9f-2228ee55ffa9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.382092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.382753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.383036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.383362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] PATH_INFO: `/v3/projects/bf9771cea58348688c22c8655a69b137/users/4f93925fc5654f4f9f426823ea67b507/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.404675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9771cea58348688c22c8655a69b137, user_id=4f93925fc5654f4f9f426823ea67b507, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.408770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.475027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 976/1980] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:19 2022] PUT /identity/v3/projects/bf9771cea58348688c22c8655a69b137/users/4f93925fc5654f4f9f426823ea67b507/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:19.479297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.570876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325c3c5-5e9f-4d80-8d84-8399eb19a79a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.575828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.576444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.576775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.577059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.577854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.580018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.602218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1e4e89fb284fed88f7e5c2558edac1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.615253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 977/1981] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:19.618952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.633454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1e4e89fb284fed88f7e5c2558edac1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.644604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8178b-626e-4a92-a811-0bcad6d809d2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.651350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1e4e89fb284fed88f7e5c2558edac1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.657502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.657502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.657804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1e4e89fb284fed88f7e5c2558edac1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.657958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.657958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] PATH_INFO: `/v3/projects/bf9771cea58348688c22c8655a69b137/users/4f93925fc5654f4f9f426823ea67b507/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.658145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1e4e89fb284fed88f7e5c2558edac1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:19.659007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9771cea58348688c22c8655a69b137, user_id=4f93925fc5654f4f9f426823ea67b507, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.662689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df10cf4-6600-4412-930d-7c9661ec7e60 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:19.685994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 978/1982] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:19 2022] PUT /identity/v3/projects/bf9771cea58348688c22c8655a69b137/users/4f93925fc5654f4f9f426823ea67b507/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:19.703020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.703020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.703020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.735385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] MFA Rules not processed for user `4f93925fc5654f4f9f426823ea67b507`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:19.736664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:19.754666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1005/1983] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:19 2022] DELETE /identity/v3/users/fe1e4e89fb284fed88f7e5c2558edac1 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:19.777782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:19.833264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 979/1984] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:19.845406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:19.846755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:19.869550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:19.908550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6cad0-ba7a-4285-992c-ac9e98e57376 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:19.912612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:19.913318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:19.913596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:19.913991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:19.916152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:19.992214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:19.993651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:20.003599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1006/1985] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:20.449089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:20.451545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:20.519710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaa986-dd75-46c6-ae70-503e2a26a8d4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:20.585434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:20.586513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:20.586513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:20.586709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] PATH_INFO: `/v3/projects/f52aa5afb5694a8f98f36b8118c7e330` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:20.587835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52aa5afb5694a8f98f36b8118c7e330)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:20.589812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:20.690985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:20.708503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52aa5afb5694a8f98f36b8118c7e330', 'request_id': 'req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:20.714011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52aa5afb5694a8f98f36b8118c7e330', 'request_id': 'req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:20.724225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52aa5afb5694a8f98f36b8118c7e330', 'request_id': 'req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:20.837102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] The token cache is being invalidate because project f52aa5afb5694a8f98f36b8118c7e330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:20.837102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:20.837102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:20.840221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 980/1986] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:20 2022] DELETE /identity/v3/projects/f52aa5afb5694a8f98f36b8118c7e330 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:20.844359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:20.856854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fab2-e3cc-4999-bd94-7432a231b87f service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:20.921240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:20.921935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:20.922573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:20.922907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] PATH_INFO: `/v3/users/a7f7df4763324e34b5cb1a6840067e87` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:20.924117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f7df4763324e34b5cb1a6840067e87)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:20.926134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:20.981794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:20.983345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:21.057683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c8e00-5af2-4663-94c7-d3f9d7bf8df7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:21.063684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:21.085867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:21.085867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:21.085867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:21.085867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:21.129535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f7df4763324e34b5cb1a6840067e87'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:21.149590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f7df4763324e34b5cb1a6840067e87'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:21.159240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f7df4763324e34b5cb1a6840067e87'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:21.160257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f7df4763324e34b5cb1a6840067e87'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:21.160928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f1c0bc37-abd5-4651-a7c2-d7ca00ffb720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f7df4763324e34b5cb1a6840067e87'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:21.175228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:21.175228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:21.191300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 981/1987] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:21.269508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1007/1988] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:20 2022] DELETE /identity/v3/users/a7f7df4763324e34b5cb1a6840067e87 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:21.663020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:21.938409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:22.352377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:22.353946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:22.378621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adc2ad-ce85-4b85-8c02-1f3f8b37cb20 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:22.450004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:22.450942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:22.451888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:22.452282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] PATH_INFO: `/v3/projects/1adccd1f23104511b5c71e7911c74d5e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:22.454434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adccd1f23104511b5c71e7911c74d5e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:22.456516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:22.591640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adccd1f23104511b5c71e7911c74d5e', 'request_id': 'req-90ffa672-4c8e-40b2-bb79-24b4067a6f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:22.598853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adccd1f23104511b5c71e7911c74d5e', 'request_id': 'req-90ffa672-4c8e-40b2-bb79-24b4067a6f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:22.605863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adccd1f23104511b5c71e7911c74d5e', 'request_id': 'req-90ffa672-4c8e-40b2-bb79-24b4067a6f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:22.724289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] The token cache is being invalidate because project 1adccd1f23104511b5c71e7911c74d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:22.727328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:22.727328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-90ffa672-4c8e-40b2-bb79-24b4067a6f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:22.727328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 982/1989] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:22 2022] DELETE /identity/v3/projects/1adccd1f23104511b5c71e7911c74d5e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:22.731962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:22.732438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:22.732713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:22.788282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:22.789789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:22.876410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1008/1990] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:22.887173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:22.887585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:22.887933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:22.928760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:22.933920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:22.939589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 983/1991] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:22.943956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:22.955649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630d172-0d4d-4789-b559-0ef634f1947c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:22.960954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:22.961645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:22.961976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:22.962311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:22.963280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:22.963727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:22.965561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:22.986880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1009/1992] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:11:22.999450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.003221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf350a0-d54d-4558-918b-ff8de4b22c72 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:23.008651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:23.009453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.009903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.010328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.011172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:23.013466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:23.182127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 984/1993] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:22 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:23.186701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.198409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e38fb-366b-4128-b69d-eabf7234a946 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:23.218255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:23.287445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:23.288225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.288509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.288778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.289435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:23.293574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:23.381720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1010/1994] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:23 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:23.384611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.405219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e3523-d6fc-43d3-9d99-ba51f89ed24d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:23.420060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:23.421002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.421585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.422347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.423314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:23.425538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:23.440255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 985/1995] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:23.443269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.454284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb1dfa-0e91-4037-b0bb-cb79297b34bc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:23.460120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:23.460740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.461017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.461286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.464284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:23.466297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6dd9a-362d-46f7-9cb0-3cdca8508ec1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:23.498317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1011/1996] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:23.500105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.531571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cabb6-aa28-42a0-a955-a6cb5c22d2a0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:23.541557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:23.542378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.542788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.543187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] PATH_INFO: `/v3/projects/f9dbec92263442f7a6497c00d9497216/users/c5154f1695ce43febcbf521258fb013c/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.580621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dbec92263442f7a6497c00d9497216, user_id=c5154f1695ce43febcbf521258fb013c, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:23.591674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:23.731751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 986/1997] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:23 2022] PUT /identity/v3/projects/f9dbec92263442f7a6497c00d9497216/users/c5154f1695ce43febcbf521258fb013c/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:23.746930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:23.747949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:23.748221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:23.784163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] MFA Rules not processed for user `c5154f1695ce43febcbf521258fb013c`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:23.785391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:23.898986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1012/1998] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:23.904194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:23.985279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:24.051604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204fc9-a276-442d-8f58-60d336366d30 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.071422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.074755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.074755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.074755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.074755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.076641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.168531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 987/1999] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:23 2022] POST /identity/v3/projects => generated 385 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:24.174732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.190180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d2b6-7cfe-49f4-b279-9b0884725f65 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.256351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.257027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.257336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.257612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.258537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.260548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.355892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1013/2000] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:24 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:24.357088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.367793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035dc51-f476-438c-8a3a-9fb0abb3649a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.372796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.373422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.373703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.373976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.375218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.376864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.387297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 988/2001] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:24.392645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.402959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18533-1a21-4eba-9c15-a8337fc15403 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.407959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.408643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.408922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.409212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.409883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.411593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43614e90-61e6-4ef6-9725-68f09adddae3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.413892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:24.422856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1014/2002] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:24.427179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.438040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b093b-8e0d-40dd-a3d3-bb2c1e8c0ea9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.443569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.444975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.444975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.444975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] PATH_INFO: `/v3/projects/4d0ee76808844c378549126c0a72502d/users/5bd2155c5b904d7c9ae923bd534ef18c/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.460770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0ee76808844c378549126c0a72502d, user_id=5bd2155c5b904d7c9ae923bd534ef18c, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.463627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.557420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 989/2003] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:24 2022] PUT /identity/v3/projects/4d0ee76808844c378549126c0a72502d/users/5bd2155c5b904d7c9ae923bd534ef18c/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:24.569753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.570133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.570364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.602137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] MFA Rules not processed for user `5bd2155c5b904d7c9ae923bd534ef18c`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:24.603564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:24.674710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1015/2004] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:24.681363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.756593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb34c7e-a728-461e-b93e-58751782c752 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.761308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.769459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.769459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.769459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.769459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.769459 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:24.850438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 990/2005] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:24 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:24.851644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:24.864197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fc7b2-3ef1-4d8c-a378-c14f1abff1af None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:24.949859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:24.950560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:24.950918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:24.951300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:24.951924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:24.954889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.049475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1016/2006] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:24 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:25.050831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:25.067896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb07d8-0b94-4156-958c-6d95ae49b9d7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:25.072677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:25.073278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.073574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.073832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.074531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:25.076166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.088400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 991/2007] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:25.092766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:25.103298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e24699-2fc8-4f97-a564-a0b697cfabdd admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:25.108118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:25.108913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.109266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.109606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] PATH_INFO: `/v3/projects/399a44647d3c4f3ea53ecadfa1c07464/users/9e8378f5fbe84ebd940078e1c79eb9c4/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.127020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399a44647d3c4f3ea53ecadfa1c07464, user_id=9e8378f5fbe84ebd940078e1c79eb9c4, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:25.129804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.240743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1017/2008] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:25 2022] PUT /identity/v3/projects/399a44647d3c4f3ea53ecadfa1c07464/users/9e8378f5fbe84ebd940078e1c79eb9c4/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:25.247599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:25.258241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355a315-1287-440d-8fa3-5a4cd6c89f7a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:25.307043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:25.317370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:25.317988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.318270 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.318543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.319249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:25.321020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b351-0d9e-4b4e-bdbd-16868b1330ac admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.331080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 992/2009] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:25.335359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:25.345945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4e112-dd97-41f5-9cfe-5c3492761478 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:25.350953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:25.351708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.351986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.352271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] PATH_INFO: `/v3/projects/399a44647d3c4f3ea53ecadfa1c07464/users/9e8378f5fbe84ebd940078e1c79eb9c4/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.354433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399a44647d3c4f3ea53ecadfa1c07464, user_id=9e8378f5fbe84ebd940078e1c79eb9c4, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:25.357118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.368757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1018/2010] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:25 2022] PUT /identity/v3/projects/399a44647d3c4f3ea53ecadfa1c07464/users/9e8378f5fbe84ebd940078e1c79eb9c4/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:25.380842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731653dd-def4-4592-a8bf-560347954749 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.381125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731653dd-def4-4592-a8bf-560347954749 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.381447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731653dd-def4-4592-a8bf-560347954749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.410952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-731653dd-def4-4592-a8bf-560347954749 None None] MFA Rules not processed for user `9e8378f5fbe84ebd940078e1c79eb9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:25.412187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-731653dd-def4-4592-a8bf-560347954749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:25.485369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 993/2011] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:25.496590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:25.498260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:25.551211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:25.566769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3ad35-d861-45fb-9aad-ad764d7fa3c9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:25.570680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:25.571332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:25.571615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:25.571912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:25.631904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:25.633404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:25.642669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1019/2012] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:28.814390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731653dd-def4-4592-a8bf-560347954749 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:28.826493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731653dd-def4-4592-a8bf-560347954749 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:28.832100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:28.832849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:28.833131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:28.833436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:28.923563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:28.925278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:28.939444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 994/2013] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:28.952265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:28.954591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:28.966330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e37acd-f014-4fde-bea3-9d5dfe2ca0fb service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:28.972212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:28.972832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:28.973128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:28.973413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:28.975226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:29.048432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:29.049908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602035-cdcc-407b-93dd-5f280db1375f service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:29.061103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1020/2014] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:29.885760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:29.887324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:30.007948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8daa66-2074-451e-a194-3e7db3776388 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:30.012879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:30.013527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:30.013799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:30.014077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] PATH_INFO: `/v3/users/d721b90127ee46d1949bbf412c74f399` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:30.015341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d721b90127ee46d1949bbf412c74f399)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:30.018374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:30.033851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:30.240434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd721b90127ee46d1949bbf412c74f399'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:30.257076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd721b90127ee46d1949bbf412c74f399'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:30.405970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd721b90127ee46d1949bbf412c74f399'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:30.410010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd721b90127ee46d1949bbf412c74f399'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:30.410010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd721b90127ee46d1949bbf412c74f399'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:30.423230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 995/2015] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:29 2022] DELETE /identity/v3/users/d721b90127ee46d1949bbf412c74f399 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:30.427996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:30.428515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:30.428515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:30.449294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:30.450498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:30.518104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1021/2016] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:30.525358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:30.530376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:30.601646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480754b-b2d6-4947-bc7c-f49a2b8c4372 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:30.605787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:30.606018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:30.606335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:30.606695 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:30.617426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:30.618885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:30.630179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 996/2017] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:30.681071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:30.682459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:30.736695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:30.763493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf4828-ca74-417f-b3e2-19429806cfb0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:30.768066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:30.768768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:30.769051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:30.769323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:30.771282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:30.883944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:30.885463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:30.904590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1022/2018] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:31.162710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:31.164381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:31.189145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88e7d-e78f-40ae-b682-98fe765e91c4 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:31.211788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:31.211788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:31.211788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:31.212189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] PATH_INFO: `/v3/projects/1393fb2e7bd84f70a533e2200226ca99` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:31.213972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1393fb2e7bd84f70a533e2200226ca99)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:31.216802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:31.338640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1393fb2e7bd84f70a533e2200226ca99', 'request_id': 'req-78d672b4-7964-4688-adc8-1142407488e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.345661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1393fb2e7bd84f70a533e2200226ca99', 'request_id': 'req-78d672b4-7964-4688-adc8-1142407488e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.352307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1393fb2e7bd84f70a533e2200226ca99', 'request_id': 'req-78d672b4-7964-4688-adc8-1142407488e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.423676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] The token cache is being invalidate because project 1393fb2e7bd84f70a533e2200226ca99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:31.423963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.424587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.426080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 997/2019] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:31 2022] DELETE /identity/v3/projects/1393fb2e7bd84f70a533e2200226ca99 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:31.429985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:31.441772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f5975-39c4-4c48-bbc4-66967cba68a1 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:31.502324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:31.502989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:31.503307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:31.503585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] PATH_INFO: `/v3/users/06d112c89331456fbf47073ccc199780` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:31.504806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d112c89331456fbf47073ccc199780)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:31.506763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:31.751587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d112c89331456fbf47073ccc199780'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.763502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d112c89331456fbf47073ccc199780'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.770956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d112c89331456fbf47073ccc199780'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.771697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d112c89331456fbf47073ccc199780'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.772297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d112c89331456fbf47073ccc199780'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:31.864223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:31.871264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1023/2020] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:31 2022] DELETE /identity/v3/users/06d112c89331456fbf47073ccc199780 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:32.365482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:32.366951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:32.386664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d672b4-7964-4688-adc8-1142407488e5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:32.397694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:32.454742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:32.455426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:32.455710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:32.456168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] PATH_INFO: `/v3/projects/7073dafd36b74983b871dbfb32d21376` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:32.457433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7073dafd36b74983b871dbfb32d21376)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:32.459425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:32.600999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7073dafd36b74983b871dbfb32d21376', 'request_id': 'req-499fb8b1-df65-43a4-aca8-bd63831da432'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.606550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7073dafd36b74983b871dbfb32d21376', 'request_id': 'req-499fb8b1-df65-43a4-aca8-bd63831da432'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.612941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7073dafd36b74983b871dbfb32d21376', 'request_id': 'req-499fb8b1-df65-43a4-aca8-bd63831da432'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.714450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] The token cache is being invalidate because project 7073dafd36b74983b871dbfb32d21376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:32.714744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.715428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.716940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 998/2021] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:32 2022] DELETE /identity/v3/projects/7073dafd36b74983b871dbfb32d21376 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:32.721101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:32.722509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:32.735687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d417b-824c-431e-9cdf-8412c439e343 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:32.794177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:32.794817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:32.795092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:32.795584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] PATH_INFO: `/v3/users/ce6cc3a2ea9241009a29ae206564dbdc` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:32.798346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6cc3a2ea9241009a29ae206564dbdc)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:32.800762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:32.973746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6cc3a2ea9241009a29ae206564dbdc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.986125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6cc3a2ea9241009a29ae206564dbdc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.993493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6cc3a2ea9241009a29ae206564dbdc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.994331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6cc3a2ea9241009a29ae206564dbdc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:32.995334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-153f6f83-4aa4-4156-b82f-f7935fa7de2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6cc3a2ea9241009a29ae206564dbdc'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.070215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1024/2022] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:32 2022] DELETE /identity/v3/users/ce6cc3a2ea9241009a29ae206564dbdc => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:33.458542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:33.462105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:33.474605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fb8b1-df65-43a4-aca8-bd63831da432 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:33.535821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:33.536366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:33.536651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:33.536933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] PATH_INFO: `/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:33.538093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2767bd7ce94449cc9011a4bdb50b18aa)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:33.540052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:33.676535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:33.678153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2767bd7ce94449cc9011a4bdb50b18aa', 'request_id': 'req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.683691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2767bd7ce94449cc9011a4bdb50b18aa', 'request_id': 'req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.692421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2767bd7ce94449cc9011a4bdb50b18aa', 'request_id': 'req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.772259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] The token cache is being invalidate because project 2767bd7ce94449cc9011a4bdb50b18aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:33.772551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.773187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-00a92e4d-7c9a-4708-84cb-e92b6a9ec972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:33.774675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 999/2023] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:33 2022] DELETE /identity/v3/projects/2767bd7ce94449cc9011a4bdb50b18aa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:33.779715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:33.780194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:33.780310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:33.801886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:33.803070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:33.863383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1025/2024] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:33.871057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:33.871394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:33.871675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:33.892109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:33.893316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:33.901834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1000/2025] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:33.906275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:33.927328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318d04e-2dc1-496c-8f59-23cac6bf5913 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:33.927328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:33.927328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:33.927328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:33.932573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:33.932573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:33.932573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:33.932573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:33.941228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1026/2026] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:11:33.945476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:33.955921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b375f-99ae-4c1c-9871-ad9ef869dbc9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:33.960758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:33.961441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:33.961728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:33.962001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:33.962638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:33.964842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.048066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1001/2027] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:33 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:34.052359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:34.058998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:34.064032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b742-628e-475e-9f2a-9ec6d9e58c54 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:34.122002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:34.122601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.122881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.123180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.123818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:34.126317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.244882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1027/2028] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:34 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:34.249083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:34.261374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ffbc8-cc3f-48c5-9fb2-85d4979198ad admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:34.266792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:34.267522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.272035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.272035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.272035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:34.272035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.281595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1002/2029] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:34.286194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:34.298924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a4b64-1f45-46d8-a68f-42f382cddd5e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:34.303915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:34.304679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.304955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.305223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.305855 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:34.307816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1a8ac-edb5-4bcf-b188-a2dd9d0485b4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.319303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1028/2030] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:34.323457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:34.333520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ae967-a45d-4659-8874-f9f3d842da49 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:34.338069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:34.338704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.338981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.339288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] PATH_INFO: `/v3/projects/0784cefbb9764465b4c9b36a4c5aecd7/users/a39c17cdfb204bea8af659548a2c476f/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.354147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0784cefbb9764465b4c9b36a4c5aecd7, user_id=a39c17cdfb204bea8af659548a2c476f, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:34.356853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.462224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1003/2031] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:34 2022] PUT /identity/v3/projects/0784cefbb9764465b4c9b36a4c5aecd7/users/a39c17cdfb204bea8af659548a2c476f/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:34.474449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.474730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.475034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.503013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] MFA Rules not processed for user `a39c17cdfb204bea8af659548a2c476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:34.504271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:34.569773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1029/2032] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:34.581377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:34.583125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:34.670599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500906f-4543-46a0-a24d-6acec2b81bce admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:34.675059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:34.675688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:34.675966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:34.676237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:34.684904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:34.686446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0b33e-f782-4904-bc13-00bedfa96b92 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:34.695955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1004/2033] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:35.360996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:35.362775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:35.439957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dff34a-9cbd-4c06-b621-ffc054568850 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:35.444628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:35.445259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:35.445537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:35.445931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] PATH_INFO: `/v3/users/2cccca11a180451c98d45f8dc6adbbf6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:35.447551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cccca11a180451c98d45f8dc6adbbf6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:35.449146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:35.465690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:35.636604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cccca11a180451c98d45f8dc6adbbf6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:35.650177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cccca11a180451c98d45f8dc6adbbf6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:35.668732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cccca11a180451c98d45f8dc6adbbf6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:35.671633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cccca11a180451c98d45f8dc6adbbf6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:35.671633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cccca11a180451c98d45f8dc6adbbf6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:35.762481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1030/2034] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:35 2022] DELETE /identity/v3/users/2cccca11a180451c98d45f8dc6adbbf6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:35.764339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:35.765497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:35.766741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:35.792381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:35.793705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:35.849984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:35.861980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1005/2035] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:35.872295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:35.873848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:35.940059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9894-0678-447a-a2e2-4e8f4187594f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:35.944014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:35.944818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:35.944942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:35.945167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:35.956088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:35.957540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:35.968176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1031/2036] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:36.454177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:36.456122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:36.469513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07581-0775-45b3-9888-f02070459f09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:36.474528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:36.475370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:36.475649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:36.475922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] PATH_INFO: `/v3/projects/b9eef1a12519460bbd0e1e7da5b88a42` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:36.477214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9eef1a12519460bbd0e1e7da5b88a42)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:36.479225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:36.602620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9eef1a12519460bbd0e1e7da5b88a42', 'request_id': 'req-6dd5b84e-5e46-4bbb-9407-abcc24815b41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:36.609432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9eef1a12519460bbd0e1e7da5b88a42', 'request_id': 'req-6dd5b84e-5e46-4bbb-9407-abcc24815b41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:36.617318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9eef1a12519460bbd0e1e7da5b88a42', 'request_id': 'req-6dd5b84e-5e46-4bbb-9407-abcc24815b41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:36.752746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:36.754156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:36.768248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623a52-d26c-406b-bb16-2cdd901d60b0 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:36.830309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:36.830947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:36.831249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:36.831528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] PATH_INFO: `/v3/users/c5154f1695ce43febcbf521258fb013c` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:36.832662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5154f1695ce43febcbf521258fb013c)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:36.834537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] The token cache is being invalidate because project b9eef1a12519460bbd0e1e7da5b88a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:36.835040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:36.838158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:36.838158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:36.838158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1006/2037] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:36 2022] DELETE /identity/v3/projects/b9eef1a12519460bbd0e1e7da5b88a42 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:36.842389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:36.854920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:36.871946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5b84e-5e46-4bbb-9407-abcc24815b41 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:36.895797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:36.895797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:36.895797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:36.895797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] PATH_INFO: `/v3/users/4f93925fc5654f4f9f426823ea67b507` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:36.896358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f93925fc5654f4f9f426823ea67b507)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:36.896358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:37.107009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:37.109299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f93925fc5654f4f9f426823ea67b507'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.120039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5154f1695ce43febcbf521258fb013c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.127589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f93925fc5654f4f9f426823ea67b507'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.136940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5154f1695ce43febcbf521258fb013c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.143819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5154f1695ce43febcbf521258fb013c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.144627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5154f1695ce43febcbf521258fb013c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.145508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5154f1695ce43febcbf521258fb013c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.217570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f93925fc5654f4f9f426823ea67b507'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.220857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f93925fc5654f4f9f426823ea67b507'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.220857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-65feec3e-76af-4f46-a8e4-d5aadfc112f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f93925fc5654f4f9f426823ea67b507'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.232799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1007/2038] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:36 2022] DELETE /identity/v3/users/4f93925fc5654f4f9f426823ea67b507 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:37.315370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1032/2039] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:36 2022] DELETE /identity/v3/users/c5154f1695ce43febcbf521258fb013c => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:37.320557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:37.321247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:37.321683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:37.347328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:37.348657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:37.420326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:37.440671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1008/2040] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:37.454176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:37.456553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:37.552919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a92283-c2ba-470f-be3e-b1bae154f450 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:37.557658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:37.558355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:37.558529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:37.565088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:37.573264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:37.575919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f970-2328-4ce7-8659-c907e71209fa service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:37.587925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1033/2041] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:37.740945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:37.742481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:37.754902 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5d123-6071-4327-98c7-10a0c47ff8e0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:37.760189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:37.760860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:37.761069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:37.761403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] PATH_INFO: `/v3/projects/bf9771cea58348688c22c8655a69b137` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:37.762536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9771cea58348688c22c8655a69b137)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:37.764496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:37.872848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9771cea58348688c22c8655a69b137', 'request_id': 'req-5df32200-caec-4c00-92fc-d29f3bd4ba8f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.879979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9771cea58348688c22c8655a69b137', 'request_id': 'req-5df32200-caec-4c00-92fc-d29f3bd4ba8f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.888234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9771cea58348688c22c8655a69b137', 'request_id': 'req-5df32200-caec-4c00-92fc-d29f3bd4ba8f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.937335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] The token cache is being invalidate because project bf9771cea58348688c22c8655a69b137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:37.937620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.939049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:37.940668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1009/2042] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:37 2022] DELETE /identity/v3/projects/bf9771cea58348688c22c8655a69b137 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:37.946501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b74a-c650-4250-8683-aca9633ef7e2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:37.947024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b74a-c650-4250-8683-aca9633ef7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:37.947024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b74a-c650-4250-8683-aca9633ef7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:37.968410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ed29b74a-c650-4250-8683-aca9633ef7e2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:37.969690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ed29b74a-c650-4250-8683-aca9633ef7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.013562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.015312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.044955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1034/2043] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:38.054433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.054719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.054991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.075035 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:38.076278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.084769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1035/2044] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:38.093074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.100028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c9966-c57a-4cb3-a201-e72c62bcfd7f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.104678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.105235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32200-caec-4c00-92fc-d29f3bd4ba8f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.105360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.105531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.105815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.106493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:38.106801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.108510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.110795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.112123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.112409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.112714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] PATH_INFO: `/v3/projects/f9dbec92263442f7a6497c00d9497216` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.113891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dbec92263442f7a6497c00d9497216)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.115898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.120911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1036/2045] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:38.125429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.134947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796329b-e626-40ee-b808-764605fd8bf5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.139595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.140171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.140445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.140712 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.141377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.143393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.232395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1037/2046] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:38 2022] POST /identity/v3/projects => generated 361 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:38.237327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.254535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dbec92263442f7a6497c00d9497216', 'request_id': 'req-48462d24-866a-47e4-8264-63a6d20d34cb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.261676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:38.266692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dbec92263442f7a6497c00d9497216', 'request_id': 'req-48462d24-866a-47e4-8264-63a6d20d34cb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.275052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dbec92263442f7a6497c00d9497216', 'request_id': 'req-48462d24-866a-47e4-8264-63a6d20d34cb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.324826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406f9c-b314-45aa-8160-ab3fd89b1d91 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.377332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] The token cache is being invalidate because project f9dbec92263442f7a6497c00d9497216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:38.381185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.381185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.381185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1010/2047] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:38 2022] DELETE /identity/v3/projects/f9dbec92263442f7a6497c00d9497216 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:38.383940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.385239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.385458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.386074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.386359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.386631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.387328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.389294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.396501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:38.400435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48462d24-866a-47e4-8264-63a6d20d34cb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.405237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.405860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.406137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.406408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] PATH_INFO: `/v3/users/5bd2155c5b904d7c9ae923bd534ef18c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.408131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd2155c5b904d7c9ae923bd534ef18c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.410080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.494718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1038/2048] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:38 2022] POST /identity/v3/users => generated 281 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:38.498754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.501181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.513554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe2278-cdc8-4b63-af27-32f13b4c2a6f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.519642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.519642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.519992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.520165 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.520817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.522424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.534073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1039/2049] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:38.537534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.548796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d9f1-3b0a-40c3-b56b-0b0ac5571db9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.553558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.555170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.555170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.555170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.555518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.556949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.569848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1040/2050] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:38.573869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.584412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb5eeb-8381-4f30-a90e-f1d826dda510 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.588992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.589620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.589894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.590162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] PATH_INFO: `/v3/projects/cac8f3d205b74262b42320f44b52a2c1/users/e89d613271604c899f895621e09645a1/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.605519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac8f3d205b74262b42320f44b52a2c1, user_id=e89d613271604c899f895621e09645a1, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.608212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ad4e-b7f7-4ec9-82b3-822da0a8c4e0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.617556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd2155c5b904d7c9ae923bd534ef18c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.632388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd2155c5b904d7c9ae923bd534ef18c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.708113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1041/2051] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:38 2022] PUT /identity/v3/projects/cac8f3d205b74262b42320f44b52a2c1/users/e89d613271604c899f895621e09645a1/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:38.717180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.717464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.717735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.721258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd2155c5b904d7c9ae923bd534ef18c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.721985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd2155c5b904d7c9ae923bd534ef18c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.722745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd2155c5b904d7c9ae923bd534ef18c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:38.737040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1011/2052] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:38 2022] DELETE /identity/v3/users/5bd2155c5b904d7c9ae923bd534ef18c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:38.751617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] MFA Rules not processed for user `e89d613271604c899f895621e09645a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:38.752837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.815761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:38.824690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1042/2053] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:38.862653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.862653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:38.935041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd39c8-07dc-4164-b756-94b87131627e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:38.939432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:38.940062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:38.940379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:38.940659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:38.949387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:38.950874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:38.969314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1012/2054] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:38.984455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:38.986079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:39.065721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3023f-0ea4-4966-abb5-5d4876f46ef9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:39.094520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:39.094520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:39.094520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:39.094520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] PATH_INFO: `/v3/users/e89d613271604c899f895621e09645a1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:39.095189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89d613271604c899f895621e09645a1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:39.095189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:39.231511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:39.235478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:39.254474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485602c-17f7-4693-86f3-1c758264827a service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:39.265011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:39.265676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:39.265997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:39.266408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] PATH_INFO: `/v3/projects/4d0ee76808844c378549126c0a72502d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:39.267719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0ee76808844c378549126c0a72502d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:39.269742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:39.400444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0ee76808844c378549126c0a72502d', 'request_id': 'req-afe09066-39be-4d20-a0d0-985559149f8d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.405889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89d613271604c899f895621e09645a1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.411800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0ee76808844c378549126c0a72502d', 'request_id': 'req-afe09066-39be-4d20-a0d0-985559149f8d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.420487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0ee76808844c378549126c0a72502d', 'request_id': 'req-afe09066-39be-4d20-a0d0-985559149f8d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.422835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89d613271604c899f895621e09645a1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.430730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89d613271604c899f895621e09645a1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.431607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89d613271604c899f895621e09645a1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.431988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a413a80d-f4fe-48b3-8363-0a42996cee40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89d613271604c899f895621e09645a1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.497823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] The token cache is being invalidate because project 4d0ee76808844c378549126c0a72502d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:39.503276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.503276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.503276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1013/2055] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:39 2022] DELETE /identity/v3/projects/4d0ee76808844c378549126c0a72502d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:39.504400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:39.505776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:39.511054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:39.559338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1043/2056] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:38 2022] DELETE /identity/v3/users/e89d613271604c899f895621e09645a1 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:39.564983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:39.566359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:39.566633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:39.583732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe09066-39be-4d20-a0d0-985559149f8d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:39.593128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:39.595881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:39.652192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:39.652828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:39.653120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:39.653396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] PATH_INFO: `/v3/users/9e8378f5fbe84ebd940078e1c79eb9c4` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:39.654554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8378f5fbe84ebd940078e1c79eb9c4)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:39.656524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:39.689369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1044/2057] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:39.703938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:39.705902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:39.726466 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:39.772315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51f4dc-cc30-4e9e-80c4-a0b3a79d18f7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:39.776767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:39.782233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:39.782233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:39.782233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:39.788353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:39.789821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:39.803273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1045/2058] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:39.881847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8378f5fbe84ebd940078e1c79eb9c4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.903689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8378f5fbe84ebd940078e1c79eb9c4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.972626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8378f5fbe84ebd940078e1c79eb9c4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.973632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8378f5fbe84ebd940078e1c79eb9c4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.974751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8378f5fbe84ebd940078e1c79eb9c4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:39.990652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1014/2059] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:39 2022] DELETE /identity/v3/users/9e8378f5fbe84ebd940078e1c79eb9c4 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:40.284321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.286878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.302659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e696f-f015-447f-b276-8fe419b16745 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.370665 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.371376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.371653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.371968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] PATH_INFO: `/v3/projects/cac8f3d205b74262b42320f44b52a2c1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.373191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac8f3d205b74262b42320f44b52a2c1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.375240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.466566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.469472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.481143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8814e0a-6a6d-4b0e-aca0-23adcfb398d4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.488203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac8f3d205b74262b42320f44b52a2c1', 'request_id': 'req-b5df01bd-e7f2-405a-a937-a906778a6f02'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.509352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:40.540450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.541337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.541602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.541870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] PATH_INFO: `/v3/projects/399a44647d3c4f3ea53ecadfa1c07464` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.542983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] RBAC: Authorizing `identity:delete_project(project_id=399a44647d3c4f3ea53ecadfa1c07464)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.544906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.558952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac8f3d205b74262b42320f44b52a2c1', 'request_id': 'req-b5df01bd-e7f2-405a-a937-a906778a6f02'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.571977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac8f3d205b74262b42320f44b52a2c1', 'request_id': 'req-b5df01bd-e7f2-405a-a937-a906778a6f02'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.583461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] The token cache is being invalidate because project cac8f3d205b74262b42320f44b52a2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:40.583746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.584401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b5df01bd-e7f2-405a-a937-a906778a6f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.585932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1046/2060] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:40 2022] DELETE /identity/v3/projects/cac8f3d205b74262b42320f44b52a2c1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:40.590529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4c81-00e7-4b28-8ac3-64a034d04a35 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.590809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4c81-00e7-4b28-8ac3-64a034d04a35 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.591130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc4c81-00e7-4b28-8ac3-64a034d04a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.611994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-37dc4c81-00e7-4b28-8ac3-64a034d04a35 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:40.613231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-37dc4c81-00e7-4b28-8ac3-64a034d04a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.621183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1047/2061] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:40.628654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.631729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.631729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.648835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:40.650033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.658742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1048/2062] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:40.663626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.671254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399a44647d3c4f3ea53ecadfa1c07464', 'request_id': 'req-8eebf0b3-dfd1-4435-b099-6c18256d646a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.674262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcae8e-57c0-4ff0-91bf-6239c1cbf146 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.677486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399a44647d3c4f3ea53ecadfa1c07464', 'request_id': 'req-8eebf0b3-dfd1-4435-b099-6c18256d646a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.683965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399a44647d3c4f3ea53ecadfa1c07464', 'request_id': 'req-8eebf0b3-dfd1-4435-b099-6c18256d646a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.707952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:40.732803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.733417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.733861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.734138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.734801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:40.735149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.736729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.748678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1049/2063] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:40.752996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] The token cache is being invalidate because project 399a44647d3c4f3ea53ecadfa1c07464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:40.752996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.752996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8eebf0b3-dfd1-4435-b099-6c18256d646a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:40.752996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1015/2064] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:40 2022] DELETE /identity/v3/projects/399a44647d3c4f3ea53ecadfa1c07464 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:40.753351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.754787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.757768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1f2c-a922-4b7b-a20a-7105d706ffed None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.758136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1f2c-a922-4b7b-a20a-7105d706ffed None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.758413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd1f2c-a922-4b7b-a20a-7105d706ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.771648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e69b4-cb74-4909-b4e1-7a677e6c6a4f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.776500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.777103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.777378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.777701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.778339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.779811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-85fd1f2c-a922-4b7b-a20a-7105d706ffed None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:40.780380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.781010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-85fd1f2c-a922-4b7b-a20a-7105d706ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.789569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1016/2065] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:40.801975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.802272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.802549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.824729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:40.825956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:40.835890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1017/2066] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:40.839761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.849655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379f094-60ce-442d-9643-c6dd2f37c6f4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.854361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.854952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.855287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.855558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.856191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:40.856573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.858140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.868729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1050/2067] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:40.869819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.869938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1018/2068] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:40.873796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:40.883568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1417dd-1713-4c9d-8dbd-3729b25c2646 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.945103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705e66c-ed99-49fb-b8d1-77626cadda62 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:40.946318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.947059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.948491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.949470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.950128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.951545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:40.952228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:40.952386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:40.952650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:40.953003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:40.954201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:40.957144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.067158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1051/2069] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:41.071306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.083062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d234f0-6994-4180-96f5-0f8d6dcb68a6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.088925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1019/2070] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:40 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:41.093187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.095181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.095803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.096075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.096338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.098619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.103956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.117754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1052/2071] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:41.122306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.128485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:41.139758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3322beb-6530-40c6-baf2-a29546da6d3e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.186539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd21a-a396-4468-9d24-8eee2e9af1ec admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.197489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.199073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.199385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.199655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.200301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.202425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.208304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.209008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.209804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.209804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.210391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.212384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.223274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1053/2072] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:41.227254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.237933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59e64-dc07-43d4-bc6d-ae41a28fbfee admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.242718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.243453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.243725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.243990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] PATH_INFO: `/v3/projects/21637fba4ff34bbe999994b890722d8c/users/fd1a6ada21d1471183e8dabfba541c78/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.260588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21637fba4ff34bbe999994b890722d8c, user_id=fd1a6ada21d1471183e8dabfba541c78, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.263345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8268c-50a1-4e1a-882d-7e4222e8d89a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.292918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1020/2073] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:41 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:41.296892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.306828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26645e96-8dcd-4f04-84bb-16c5d4347a94 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.312231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.319283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.319283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.319283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.319283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.319283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.327145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1021/2074] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:41.333029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.339312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:41.343366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632378d5-a77f-46ae-8b12-cdd0fad7b471 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.346102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1054/2075] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:41 2022] PUT /identity/v3/projects/21637fba4ff34bbe999994b890722d8c/users/fd1a6ada21d1471183e8dabfba541c78/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:41.357023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.357304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.357625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.392306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] MFA Rules not processed for user `fd1a6ada21d1471183e8dabfba541c78`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:41.395794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:41.423522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.424203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.424484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.424757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.425413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.427534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.440586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1022/2076] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:41.444247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.458048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab539c-1b86-4125-a4cb-7b5cfa966bf6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.463863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.464608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.465320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.465666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] PATH_INFO: `/v3/projects/ed89dd1253354dc3b4b727bb8f038297/users/94b84581bd1d479a8ee559439b9e2d3a/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.483800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed89dd1253354dc3b4b727bb8f038297, user_id=94b84581bd1d479a8ee559439b9e2d3a, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.489541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cf1bc-b63f-4bb4-9ae0-90f513ff3c6e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.495775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1055/2077] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:41.508527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.509930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:41.585010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1023/2078] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:41 2022] PUT /identity/v3/projects/ed89dd1253354dc3b4b727bb8f038297/users/94b84581bd1d479a8ee559439b9e2d3a/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:41.585229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c0355-cd9d-43a2-a345-033959eeb83e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.588058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.588333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.588568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.622703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] MFA Rules not processed for user `94b84581bd1d479a8ee559439b9e2d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:41.624554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:41.639666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.640279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.640550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.640874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.703399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:41.710894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1024/2079] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:41.719879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.719879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.722857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.723593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:41.724258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:41.736219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1056/2080] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:41.737540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11679372-ee78-4ee1-8722-bd9a4909316a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.742335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.747743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.747743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.747743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.752987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.754467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60703dd-e8b6-4740-84a0-57bc29bedf79 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:41.763944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1025/2081] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:41.791654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:41.862963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe14a-be99-4e72-8dfa-06400e1c04ac service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:41.869813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:41.870894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:41.871374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:41.871963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] PATH_INFO: `/v3/users/94b84581bd1d479a8ee559439b9e2d3a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:41.873755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b84581bd1d479a8ee559439b9e2d3a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:41.876620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:42.102990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b84581bd1d479a8ee559439b9e2d3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:42.117143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b84581bd1d479a8ee559439b9e2d3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:42.124055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b84581bd1d479a8ee559439b9e2d3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:42.124798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b84581bd1d479a8ee559439b9e2d3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:42.126241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b84581bd1d479a8ee559439b9e2d3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:42.254970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1057/2082] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:41 2022] DELETE /identity/v3/users/94b84581bd1d479a8ee559439b9e2d3a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:42.260888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:42.262035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:42.262721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:42.305541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:42.305541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:42.384837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1026/2083] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:42.396320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:42.398321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:42.481833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2acbb-70f5-4681-b263-5be9e78dc330 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:42.486486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:42.489509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:42.489509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:42.489509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:42.498278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:42.499631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff12091-bf85-4733-8ca5-09e8cd3aff53 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:42.524146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1058/2084] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:43.402441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:43.404532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:43.425341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72bea-ed25-4e1b-9f39-542a05560025 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:43.431004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:43.433221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:43.433221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:43.433221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] PATH_INFO: `/v3/projects/ed89dd1253354dc3b4b727bb8f038297` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:43.435134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed89dd1253354dc3b4b727bb8f038297)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:43.439779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:43.551135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed89dd1253354dc3b4b727bb8f038297', 'request_id': 'req-7aaf5a95-2912-47cf-950c-30b1eb18e16e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:43.560668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed89dd1253354dc3b4b727bb8f038297', 'request_id': 'req-7aaf5a95-2912-47cf-950c-30b1eb18e16e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:43.575808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed89dd1253354dc3b4b727bb8f038297', 'request_id': 'req-7aaf5a95-2912-47cf-950c-30b1eb18e16e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:43.675375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] The token cache is being invalidate because project ed89dd1253354dc3b4b727bb8f038297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:43.675760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:43.676424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7aaf5a95-2912-47cf-950c-30b1eb18e16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:43.677929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1027/2085] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:43 2022] DELETE /identity/v3/projects/ed89dd1253354dc3b4b727bb8f038297 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:45.353248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dea05-6f55-45f7-9201-cca7e82ce0f2 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.353836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dea05-6f55-45f7-9201-cca7e82ce0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.353940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096dea05-6f55-45f7-9201-cca7e82ce0f2 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.355295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1059/2086] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:11:45.382325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb43be-7d4f-41dd-a173-fcf42b40e0e9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.382829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb43be-7d4f-41dd-a173-fcf42b40e0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.382923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb43be-7d4f-41dd-a173-fcf42b40e0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.406848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0bcb43be-7d4f-41dd-a173-fcf42b40e0e9 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:45.408650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0bcb43be-7d4f-41dd-a173-fcf42b40e0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:45.472070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1028/2087] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:45.485876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8696-970c-4c06-b12b-39858c443349 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.486164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8696-970c-4c06-b12b-39858c443349 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.486508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8696-970c-4c06-b12b-39858c443349 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.487895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1060/2088] 10.222.0.46 () {54 vars in 961 bytes} [Thu Sep 1 09:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:11:45.493911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.494204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.497326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.502820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f8696-970c-4c06-b12b-39858c443349 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:45.504484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-907f8696-970c-4c06-b12b-39858c443349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:45.507348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:45.528131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:45.529765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:45.530894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:45.539600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1029/2089] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Sep 1 09:11:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:45.548350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:45.559510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17009a6-e1fe-4b06-8b38-3109fdfb1a80 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:45.566246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:45.566875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.567193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.567468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.576314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:45.578596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:45.585345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f8696-970c-4c06-b12b-39858c443349 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:45.588741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1030/2090] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:45.590009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:45.590632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:45.590905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:45.591204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] PATH_INFO: `/v3/users/f6d6f82f9aa6447591cdd55d2a040ea4` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:45.592327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d6f82f9aa6447591cdd55d2a040ea4)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:45.594223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:45.746284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d6f82f9aa6447591cdd55d2a040ea4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:45.757997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d6f82f9aa6447591cdd55d2a040ea4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:45.765744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d6f82f9aa6447591cdd55d2a040ea4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:45.767331 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d6f82f9aa6447591cdd55d2a040ea4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:45.768160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d6f82f9aa6447591cdd55d2a040ea4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:45.818180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1061/2091] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:45 2022] DELETE /identity/v3/users/f6d6f82f9aa6447591cdd55d2a040ea4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:46.072037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.073993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.090118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52386d-e289-448f-883f-5312325d2514 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.148640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.149258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.149537 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.149811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] PATH_INFO: `/v3/users/fd1a6ada21d1471183e8dabfba541c78` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.150907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1a6ada21d1471183e8dabfba541c78)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.152890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.174664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.179676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.195448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89751-7286-4130-8218-e2d6a9875af4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.200652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.201288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.201561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.201825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] PATH_INFO: `/v3/projects/e27cafb5f07e449295c45c033ee62256` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.202976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27cafb5f07e449295c45c033ee62256)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.209191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.392749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1a6ada21d1471183e8dabfba541c78'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.393285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27cafb5f07e449295c45c033ee62256', 'request_id': 'req-337c55ae-1a48-483a-96d8-2ad91c04c05b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.404585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1a6ada21d1471183e8dabfba541c78'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.452295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27cafb5f07e449295c45c033ee62256', 'request_id': 'req-337c55ae-1a48-483a-96d8-2ad91c04c05b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.460923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27cafb5f07e449295c45c033ee62256', 'request_id': 'req-337c55ae-1a48-483a-96d8-2ad91c04c05b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.473561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] The token cache is being invalidate because project e27cafb5f07e449295c45c033ee62256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:46.473921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.474526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-337c55ae-1a48-483a-96d8-2ad91c04c05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.476128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1062/2092] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:46 2022] DELETE /identity/v3/projects/e27cafb5f07e449295c45c033ee62256 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:46.480669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0f0fe-39b3-4d6b-b828-660f2dc061f8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.480950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0f0fe-39b3-4d6b-b828-660f2dc061f8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.482701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0f0fe-39b3-4d6b-b828-660f2dc061f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.498126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1a6ada21d1471183e8dabfba541c78'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.498742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1a6ada21d1471183e8dabfba541c78'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.499370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10761323-8989-4dcd-ab2a-8dd8bbd4711d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1a6ada21d1471183e8dabfba541c78'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:46.500997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-18c0f0fe-39b3-4d6b-b828-660f2dc061f8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:46.502221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-18c0f0fe-39b3-4d6b-b828-660f2dc061f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.513664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1031/2093] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:11:46 2022] DELETE /identity/v3/users/fd1a6ada21d1471183e8dabfba541c78 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:46.518799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.519113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.519390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.520952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:46.540402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:46.541879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.577249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1063/2094] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:11:46.585565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.586024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.586303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.614249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:46.615608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.627575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1064/2095] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:46.627779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1032/2096] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:46.632448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.639955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.642161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:46.643261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958af05-81fd-4e6d-bc0b-4e9dd0a73e66 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.649384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:46.650015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.650621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.650888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.651439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.652142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:11:46.652419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.654388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.668152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1065/2097] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:11:46.672630 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.682814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2928-43f1-4bd3-923d-37958f5d2112 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.687638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.689669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.689995 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.690293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.690965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.693159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.707650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553f8a6-4f98-48c0-a2bc-967e538fe076 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.711388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.712124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.712506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.712877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.722648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.724141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.733753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1033/2098] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:11:46.800878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1066/2099] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:46 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:46.802026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.812161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0eebc7-be81-49f1-bac5-13ee94f29a52 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:46.870547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:46.871194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:46.871478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:46.871749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:46.872375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:46.874312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:46.981740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1034/2100] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:46 2022] POST /identity/v3/users => generated 282 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:46.982985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:46.995789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec8dc-e17b-47f2-9530-4ab8e62461c9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.002933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.003648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.003929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.004201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.004848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.006478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.020218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1067/2101] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:47.024365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.034893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa718f-f497-4de5-9e21-43ec8fe160dd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.039699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.040395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.040736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.041353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.042083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.043774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.054860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1035/2102] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:47.058961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.075667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf0d75-2efb-4c9b-a38c-786b71ace2a6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.081117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.081860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.082215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.082968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] PATH_INFO: `/v3/projects/9d18699e2ce54194a867d8f4f0e3323b/users/730b37237d5543e6ac5febf64374aa7b/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.105883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d18699e2ce54194a867d8f4f0e3323b, user_id=730b37237d5543e6ac5febf64374aa7b, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.108603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf71c5-7c57-4223-a95a-0a8f72250405 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.191411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.192552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1068/2103] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:47 2022] PUT /identity/v3/projects/9d18699e2ce54194a867d8f4f0e3323b/users/730b37237d5543e6ac5febf64374aa7b/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:47.193516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:47.202757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.203048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.203358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.226009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb73ef-7e29-4bf7-8c8e-e8815d18d646 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.240207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] MFA Rules not processed for user `730b37237d5543e6ac5febf64374aa7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:47.241522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:47.293688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.294343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.294644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.294914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] PATH_INFO: `/v3/projects/21637fba4ff34bbe999994b890722d8c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.296166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=21637fba4ff34bbe999994b890722d8c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.298635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.322218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1069/2104] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:47.327015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.338382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ba452-6cbe-4a64-90cb-090a686ddba3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.343034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.343678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.343958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.344225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.345770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.347738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.390439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:47.405352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21637fba4ff34bbe999994b890722d8c', 'request_id': 'req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:47.411703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21637fba4ff34bbe999994b890722d8c', 'request_id': 'req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:47.418620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21637fba4ff34bbe999994b890722d8c', 'request_id': 'req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:47.441950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1070/2105] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:11:47 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:11:47.445969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.459899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:47.508977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] The token cache is being invalidate because project 21637fba4ff34bbe999994b890722d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:11:47.509257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:47.509923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:11:47.510412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e81b7-f86a-4496-bf88-7aacaecca5c4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.511465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1036/2106] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:11:47 2022] DELETE /identity/v3/projects/21637fba4ff34bbe999994b890722d8c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:47.515381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.515998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.516326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.516596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.517405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.519420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.607566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1071/2107] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:11:47 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:11:47.612498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.616162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:47.687125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29330c-0501-4ae4-a37c-aa7d39fa6aaa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.693420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.694458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.694745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.695073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.696368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.697999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.713232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1037/2108] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:11:47.714675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.725492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b41a4-2e8f-4077-90ef-3652c55e5abc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.730166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.730803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.731083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.731397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] PATH_INFO: `/v3/projects/5792667a53a54498b29e12da5c55e5c5/users/44cca4713f3c412784d1c991a0b70190/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.745920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5792667a53a54498b29e12da5c55e5c5, user_id=44cca4713f3c412784d1c991a0b70190, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.748621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.819317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1072/2109] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:47 2022] PUT /identity/v3/projects/5792667a53a54498b29e12da5c55e5c5/users/44cca4713f3c412784d1c991a0b70190/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:11:47.823302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.834038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ffb3-46b3-4d77-b6e8-b2063d1f26f7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.894688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.895319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.895601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.895876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.896519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.898138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50b4b-e097-41c7-87f7-442400603cda admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.907975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1038/2110] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:11:47.912294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:47.922797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c3682-6bfb-42b7-91e3-bd102d122e5f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:47.927583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:47.928220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.928502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.928780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] PATH_INFO: `/v3/projects/5792667a53a54498b29e12da5c55e5c5/users/44cca4713f3c412784d1c991a0b70190/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.930753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5792667a53a54498b29e12da5c55e5c5, user_id=44cca4713f3c412784d1c991a0b70190, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:47.933432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:47.945003 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1073/2111] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:11:47 2022] PUT /identity/v3/projects/5792667a53a54498b29e12da5c55e5c5/users/44cca4713f3c412784d1c991a0b70190/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:11:47.957148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:47.957433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:47.957711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:47.986165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] MFA Rules not processed for user `44cca4713f3c412784d1c991a0b70190`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:11:47.987328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:48.027312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:48.056970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1039/2112] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:11:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:11:48.068850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:48.070524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:48.128615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2959f-f1cb-4ce8-ae92-6b93c5196b6b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:48.132290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:48.132896 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:48.133175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:48.133449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:48.143153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:48.144580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:48.154294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1074/2113] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:48.402029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:11:48.403551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:48.421394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661040e-c14a-4ea2-b7ff-f6b72c22766e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:11:48.426173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:11:48.426781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:11:48.427060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:11:48.427380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:11:48.429381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:11:48.493922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:11:48.495389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af4ae-7f1b-492a-ad9d-3291c733025e service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:11:48.505576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1040/2114] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:11:58.149903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=64,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:11:58.500244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=40 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:04.581414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:04.643122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff333-2c95-4e1d-bc5c-b84c78c9a5bc service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:04.647963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:04.648573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:04.660548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:04.660548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] PATH_INFO: `/v3/users/730b37237d5543e6ac5febf64374aa7b` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:04.660548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730b37237d5543e6ac5febf64374aa7b)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:04.660548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:04.810096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730b37237d5543e6ac5febf64374aa7b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:04.822479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730b37237d5543e6ac5febf64374aa7b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:04.829799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730b37237d5543e6ac5febf64374aa7b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:04.831649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730b37237d5543e6ac5febf64374aa7b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:04.832846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730b37237d5543e6ac5febf64374aa7b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:04.921860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1075/2115] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:04 2022] DELETE /identity/v3/users/730b37237d5543e6ac5febf64374aa7b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:04.928450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:04.928450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:04.928450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:04.947668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:04.948933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:05.008837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1041/2116] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:05.019168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:05.020756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:05.079753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691dd25-375d-439d-9114-656e60482cd3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:05.084113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:05.084716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:05.084997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:05.085270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:05.094170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:05.095627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:05.104952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1076/2117] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:05.523396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:05.524893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:05.543861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fe27e-33a3-40e6-aab6-43f3126a2019 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:05.556581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:05.557226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:05.557507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:05.557781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] PATH_INFO: `/v3/projects/9d18699e2ce54194a867d8f4f0e3323b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:05.559201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d18699e2ce54194a867d8f4f0e3323b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:05.561356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:05.694337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d18699e2ce54194a867d8f4f0e3323b', 'request_id': 'req-cc6dec1b-e09f-4998-9c00-c82dfefe3635'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:05.699869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d18699e2ce54194a867d8f4f0e3323b', 'request_id': 'req-cc6dec1b-e09f-4998-9c00-c82dfefe3635'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:05.706181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d18699e2ce54194a867d8f4f0e3323b', 'request_id': 'req-cc6dec1b-e09f-4998-9c00-c82dfefe3635'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:05.795268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] The token cache is being invalidate because project 9d18699e2ce54194a867d8f4f0e3323b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:05.795552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:05.796175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:05.797712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1042/2118] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:05 2022] DELETE /identity/v3/projects/9d18699e2ce54194a867d8f4f0e3323b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:05.802287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:05.811994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9bb7f-274b-4361-ae36-972e90beb344 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:05.829884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:05.868208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:05.868830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:05.869105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:05.869378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] PATH_INFO: `/v3/users/44cca4713f3c412784d1c991a0b70190` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:05.870492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] RBAC: Authorizing `identity:delete_user(user_id=44cca4713f3c412784d1c991a0b70190)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:05.872455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:06.135706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44cca4713f3c412784d1c991a0b70190'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.148871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44cca4713f3c412784d1c991a0b70190'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.155914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44cca4713f3c412784d1c991a0b70190'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.156640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44cca4713f3c412784d1c991a0b70190'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.157251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d0a78084-2bce-4042-aa4d-3186c1c7a57e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44cca4713f3c412784d1c991a0b70190'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.218500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1077/2119] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:05 2022] DELETE /identity/v3/users/44cca4713f3c412784d1c991a0b70190 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:06.648394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:06.649921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:06.663813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dec1b-e09f-4998-9c00-c82dfefe3635 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:06.720825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:06.722021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:06.722299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:06.722548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] PATH_INFO: `/v3/projects/5792667a53a54498b29e12da5c55e5c5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:06.726787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5792667a53a54498b29e12da5c55e5c5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:06.726787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:06.832132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:06.849972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5792667a53a54498b29e12da5c55e5c5', 'request_id': 'req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.855505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5792667a53a54498b29e12da5c55e5c5', 'request_id': 'req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.862022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5792667a53a54498b29e12da5c55e5c5', 'request_id': 'req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.950822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] The token cache is being invalidate because project 5792667a53a54498b29e12da5c55e5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:06.951131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.951790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d0a6ff78-968c-42ab-a8c7-6d57fb2db9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:06.956834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1043/2120] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:06 2022] DELETE /identity/v3/projects/5792667a53a54498b29e12da5c55e5c5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:06.958877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:06.959412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:06.959412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:06.982072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:06.983354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:07.046960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1078/2121] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:12:07.054756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.055036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.055361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.078526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:07.079770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:07.089785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1044/2122] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:12:07.093940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.104928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d803c4-ccba-4134-89dc-61f67746b66b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.110166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.110765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.111041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.111334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.111971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:07.112267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.113872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.131424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1079/2123] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:12:07.135591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.145748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4e86e-aaa8-4057-8208-24219de52055 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.150518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.151162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.151445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.151717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.152365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.154332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.268655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1045/2124] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:07.270501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.284347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f80f6-f881-4343-b511-271be7532f4a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.311705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:07.343576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.344242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.344516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.344978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.345607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.347583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.450848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1080/2125] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:07.454427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.465700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52e18e-8090-497e-963d-5c69eb343e7a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.470408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.471013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.471351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.473795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.473795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.474186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.485145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1046/2126] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:07.489955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.500129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ab9e-c06d-432d-8922-763aba8a4950 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.505525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.506130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.506411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.506693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.507376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.509007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58957b78-a078-444b-8ac8-5338a99b5612 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.520570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1081/2127] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:07.525114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.536110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd2365-6f2d-4fba-9314-5ff68864b202 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.541318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.542273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.542552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.542858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] PATH_INFO: `/v3/projects/52b4f795848b4dfb84925c495a1aa9cc/users/4436a4228970424993bcbf407e228073/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.559488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b4f795848b4dfb84925c495a1aa9cc, user_id=4436a4228970424993bcbf407e228073, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.562202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.642052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1047/2128] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:07 2022] PUT /identity/v3/projects/52b4f795848b4dfb84925c495a1aa9cc/users/4436a4228970424993bcbf407e228073/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:07.654203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.654565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.654906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.684020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] MFA Rules not processed for user `4436a4228970424993bcbf407e228073`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:07.685339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:07.749661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1082/2129] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:07.754999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.818083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6594e9-be38-4a0e-8608-38610da111ed admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.822799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.823453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.823735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.824010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:07.824653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:07.826563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:07.923578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1048/2130] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:07.928034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:07.939036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153387e9-570d-4145-b7f4-6a64db7275e7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:07.998269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:07.998912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:07.999305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:07.999476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.000101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.002092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.075978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1083/2131] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:07 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:08.079987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:08.090765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88aa3-ab2d-4e3b-9070-1e9983addf69 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:08.095457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:08.096045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.096332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.096604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.097239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.099745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.110416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1049/2132] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:08.115022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:08.126642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65482bc-f345-4be3-b88f-8e5afd2e1be4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:08.137870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:08.138515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.138788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.139133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] PATH_INFO: `/v3/projects/848178a6362e46e5a376b73acc1f67a6/users/b2dcbd5b98c74607b382bac11e27d157/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.156611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848178a6362e46e5a376b73acc1f67a6, user_id=b2dcbd5b98c74607b382bac11e27d157, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.159433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.231932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1084/2133] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:08 2022] PUT /identity/v3/projects/848178a6362e46e5a376b73acc1f67a6/users/b2dcbd5b98c74607b382bac11e27d157/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:08.235767 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:08.246595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a3e43-c3dc-4425-91f2-a63219e4884a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:08.266268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:08.304226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:08.304822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.305099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.305369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.305997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.307612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494ce1f-6c9a-4a55-84d7-c66e71503e77 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.317292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1050/2134] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:08.321675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:08.327177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:08.332366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a6799-a3da-47cc-8b16-febe7a22e343 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:08.337188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:08.337853 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.338127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.338395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] PATH_INFO: `/v3/projects/848178a6362e46e5a376b73acc1f67a6/users/b2dcbd5b98c74607b382bac11e27d157/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.340302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=848178a6362e46e5a376b73acc1f67a6, user_id=b2dcbd5b98c74607b382bac11e27d157, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.342970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.353836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1085/2135] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:08 2022] PUT /identity/v3/projects/848178a6362e46e5a376b73acc1f67a6/users/b2dcbd5b98c74607b382bac11e27d157/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:08.364796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.365160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.365496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.392490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] MFA Rules not processed for user `b2dcbd5b98c74607b382bac11e27d157`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:08.393678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:08.461699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1051/2136] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:08.472237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:08.474560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:08.531900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7212da-57f7-440c-9253-aff94305a6cf admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:08.537153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:08.537153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:08.537153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:08.537375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:08.546610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:08.548056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45950a8-1ebe-45f1-8621-bc31892cebd0 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:08.557724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1086/2137] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:11.305596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:11.379445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e838aab-4713-4a85-968c-4ddcc4b83f31 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:11.382836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:11.383492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:11.383773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:11.384045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] PATH_INFO: `/v3/users/4436a4228970424993bcbf407e228073` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:11.385151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4436a4228970424993bcbf407e228073)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:11.387069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:11.542625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4436a4228970424993bcbf407e228073'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:11.543796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:11.560047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4436a4228970424993bcbf407e228073'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:11.659201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4436a4228970424993bcbf407e228073'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:11.659818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4436a4228970424993bcbf407e228073'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:11.660422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4436a4228970424993bcbf407e228073'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:11.674393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1052/2138] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:11 2022] DELETE /identity/v3/users/4436a4228970424993bcbf407e228073 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:11.679422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:11.679904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:11.680004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:11.699144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:11.700325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:11.757587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1087/2139] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:11.767166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:11.768596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:11.824838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3741-e8f0-49fb-98ce-67723f8666dc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:11.828378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:11.828969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:11.829250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:11.829521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:11.838903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:11.840379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:11.849782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1053/2140] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:12.248722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:12.250147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:12.263258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb271d-3c3b-4ad7-bced-0ca3ae0f6eb8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:12.269308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:12.269946 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:12.270229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:12.270502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] PATH_INFO: `/v3/projects/52b4f795848b4dfb84925c495a1aa9cc` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:12.271907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b4f795848b4dfb84925c495a1aa9cc)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:12.273909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:12.380099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b4f795848b4dfb84925c495a1aa9cc', 'request_id': 'req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.447691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b4f795848b4dfb84925c495a1aa9cc', 'request_id': 'req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.454703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b4f795848b4dfb84925c495a1aa9cc', 'request_id': 'req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.465361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] The token cache is being invalidate because project 52b4f795848b4dfb84925c495a1aa9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:12.465646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.466268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.467801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1088/2141] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:12 2022] DELETE /identity/v3/projects/52b4f795848b4dfb84925c495a1aa9cc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:12.471776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:12.482048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b157cf-c73e-4e4d-9022-68db2eb73658 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:12.535689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:12.536296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:12.536572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:12.537456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] PATH_INFO: `/v3/users/b2dcbd5b98c74607b382bac11e27d157` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:12.538566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2dcbd5b98c74607b382bac11e27d157)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:12.540643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:12.712053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2dcbd5b98c74607b382bac11e27d157'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.723628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2dcbd5b98c74607b382bac11e27d157'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.727815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:12.796144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2dcbd5b98c74607b382bac11e27d157'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.796786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2dcbd5b98c74607b382bac11e27d157'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.797393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e1900b2b-21e0-4247-bf18-60317399a165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2dcbd5b98c74607b382bac11e27d157'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:12.811611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1054/2142] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:12 2022] DELETE /identity/v3/users/b2dcbd5b98c74607b382bac11e27d157 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:13.216449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.217975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:13.231576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0b24a-79ff-4445-a7ba-7b6b5252e58a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.248891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:13.290304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.290930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.291251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.291528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] PATH_INFO: `/v3/projects/848178a6362e46e5a376b73acc1f67a6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.292892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848178a6362e46e5a376b73acc1f67a6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.294805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.367569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848178a6362e46e5a376b73acc1f67a6', 'request_id': 'req-a136ad33-e0f2-47f7-9351-4106c3946809'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:13.423553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848178a6362e46e5a376b73acc1f67a6', 'request_id': 'req-a136ad33-e0f2-47f7-9351-4106c3946809'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:13.430463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848178a6362e46e5a376b73acc1f67a6', 'request_id': 'req-a136ad33-e0f2-47f7-9351-4106c3946809'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:13.442796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] The token cache is being invalidate because project 848178a6362e46e5a376b73acc1f67a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:13.443792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:13.445256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a136ad33-e0f2-47f7-9351-4106c3946809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:13.446961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1089/2143] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:13 2022] DELETE /identity/v3/projects/848178a6362e46e5a376b73acc1f67a6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:13.451743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.455621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.455621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.472891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:13.474079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:13.532805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1055/2144] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:13.540469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.540752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.541022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.560978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:13.562179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:13.570744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1090/2145] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:13.574943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.585196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec616155-80b0-4806-9dd9-a9ef5ac8f90a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.589711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.590292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.590567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.590838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.591511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:13.591812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.593411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.604697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1056/2146] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:12:13.608885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.618453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09eba7-876f-4a37-996c-0db70a9999eb None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.623130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.623720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.623998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.624268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.624924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.626825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.710502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1091/2147] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:13 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:13.714655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.724794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725fc98-91de-4780-a19f-e13f3e7cd4cd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.780753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.781350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.781630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.781900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.782516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.784519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.852408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1057/2148] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:13 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:12:13.856724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.867959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ca85c-99cc-472d-80d4-2fb32910e5ec admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.872885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.875775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.875775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.875775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.875775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.876271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.886656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1092/2149] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:12:13.890608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.900797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e93719-0b23-4099-8491-a1410af71de2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.905420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.906005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.906287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.906561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.907236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.908857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebb5f0-9af1-4fca-ab86-fd4490c1fb28 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:13.919963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1058/2150] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:13.924010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:13.934566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43da78e-4c7e-422b-bd3e-da8b903af34a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:13.939169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:13.939800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:13.940076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:13.940348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] PATH_INFO: `/v3/projects/f85e24472b654359954478b209412081/users/9743889e750543b68411a10a791ecda3/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:13.955244 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85e24472b654359954478b209412081, user_id=9743889e750543b68411a10a791ecda3, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:13.957883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.018766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1093/2151] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:13 2022] PUT /identity/v3/projects/f85e24472b654359954478b209412081/users/9743889e750543b68411a10a791ecda3/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:14.030059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.030346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.030649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.059921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] MFA Rules not processed for user `9743889e750543b68411a10a791ecda3`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:14.061095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:14.102497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:14.124866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1059/2152] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:14.129627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.175820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:14.193557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a081a7-2500-4bf8-87ff-097d8a88b23c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.199212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.199829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.200107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.200376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.200994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.202884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.265619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1094/2153] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:14 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:14.269830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.280553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247c7b2-b1ee-48b4-94cc-b634a117a88b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.335868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.336389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.336665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.336939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.337559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.339507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.414590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1060/2154] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:14 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:12:14.418769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.429180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ded75-6134-4dfb-87a0-4db92f990bf4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.433912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.434505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.434787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.435089 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.435756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.437349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.447577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1095/2155] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:14.451821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.462215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040ac9-fadd-40f1-8ba9-337253bc5ce7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.466829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.467489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.467765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.468031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] PATH_INFO: `/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9/users/28f6953798294c1db5400871fc50a2e5/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.483616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5eab20e8f4bc3a9b703553f74bec9, user_id=28f6953798294c1db5400871fc50a2e5, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.486286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.551321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1061/2156] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:14 2022] PUT /identity/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9/users/28f6953798294c1db5400871fc50a2e5/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:14.556143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.629423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898f9e2-5439-404d-9ec6-be1a742748ac admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.632884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.633494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.633773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.634045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.634721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.636389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9c2cf-9c26-494f-91f0-5b09fa4dcf24 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.646434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1096/2157] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:14.650626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.660953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49da7c4-2161-4ce0-8aac-10398169e18a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.665588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.666219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.666495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.666764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] PATH_INFO: `/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9/users/28f6953798294c1db5400871fc50a2e5/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.668947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5eab20e8f4bc3a9b703553f74bec9, user_id=28f6953798294c1db5400871fc50a2e5, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.671674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.682081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1062/2158] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:14 2022] PUT /identity/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9/users/28f6953798294c1db5400871fc50a2e5/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:14.691904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.692192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.692469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.721924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] MFA Rules not processed for user `28f6953798294c1db5400871fc50a2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:14.723160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:14.790225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1097/2159] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:14.801532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.803965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:14.860444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756fcb-ddb2-4a37-becc-cb6d5619edd3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:14.864058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:14.864649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:14.865557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:14.866048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:14.874896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:14.876348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9895266-eddc-42a5-a747-9e13754e5e58 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:14.885884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1063/2160] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:14.957119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:14.998510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:15.018790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ee2aa-16ac-4b82-8bb5-f9fe2ca4c780 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:15.023622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:15.024249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:15.024529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:15.024825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] PATH_INFO: `/v3/users/9743889e750543b68411a10a791ecda3` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:15.025964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9743889e750543b68411a10a791ecda3)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:15.027950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:15.181966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9743889e750543b68411a10a791ecda3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.194172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9743889e750543b68411a10a791ecda3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.201049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9743889e750543b68411a10a791ecda3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.201734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9743889e750543b68411a10a791ecda3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.202339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9743889e750543b68411a10a791ecda3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.276892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1098/2161] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:14 2022] DELETE /identity/v3/users/9743889e750543b68411a10a791ecda3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:15.281416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:15.281703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:15.281976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:15.301955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:15.303326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:15.334058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:15.360744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1064/2162] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:15.370406 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:15.371971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:15.429463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197234a-39b8-4fef-8ad2-2d72204387e6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:15.433335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:15.433868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:15.434140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:15.434407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:15.443341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:15.444803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:15.454006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1099/2163] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:15.827177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:15.828661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:15.841541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af9b9e-5c33-4c36-984d-4ed786462a73 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:15.846344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:15.847059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:15.847362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:15.847633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] PATH_INFO: `/v3/projects/f85e24472b654359954478b209412081` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:15.848836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85e24472b654359954478b209412081)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:15.850938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:15.968742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85e24472b654359954478b209412081', 'request_id': 'req-4f17bf36-dff3-41d3-b9c8-d78709d70329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.974218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85e24472b654359954478b209412081', 'request_id': 'req-4f17bf36-dff3-41d3-b9c8-d78709d70329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:15.980561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85e24472b654359954478b209412081', 'request_id': 'req-4f17bf36-dff3-41d3-b9c8-d78709d70329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.070598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] The token cache is being invalidate because project f85e24472b654359954478b209412081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:16.070877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.071527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.073014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1065/2164] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:15 2022] DELETE /identity/v3/projects/f85e24472b654359954478b209412081 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:16.077122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:16.086862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6a63b-0b84-43c0-8b48-052029f9ef01 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:16.142464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:16.143083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:16.143399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:16.143677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] PATH_INFO: `/v3/users/28f6953798294c1db5400871fc50a2e5` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:16.144783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f6953798294c1db5400871fc50a2e5)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:16.146699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:16.148246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:16.347309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f6953798294c1db5400871fc50a2e5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.360051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f6953798294c1db5400871fc50a2e5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.367752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f6953798294c1db5400871fc50a2e5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.368515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f6953798294c1db5400871fc50a2e5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.369130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-70035103-6265-44ef-a6d1-72fbe1f6751c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f6953798294c1db5400871fc50a2e5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:16.448106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1100/2165] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:16 2022] DELETE /identity/v3/users/28f6953798294c1db5400871fc50a2e5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:16.921969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:16.923583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:16.937475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17bf36-dff3-41d3-b9c8-d78709d70329 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.019871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.020522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.020805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.021082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] PATH_INFO: `/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.022542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a5eab20e8f4bc3a9b703553f74bec9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.025215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.153360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a5eab20e8f4bc3a9b703553f74bec9', 'request_id': 'req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:17.159173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a5eab20e8f4bc3a9b703553f74bec9', 'request_id': 'req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:17.167191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a5eab20e8f4bc3a9b703553f74bec9', 'request_id': 'req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:17.169169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:17.276450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] The token cache is being invalidate because project c3a5eab20e8f4bc3a9b703553f74bec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:17.276745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:17.277432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e08726b-3d7b-4ad9-9d15-cfcd1454abdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:17.279169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1066/2166] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:16 2022] DELETE /identity/v3/projects/c3a5eab20e8f4bc3a9b703553f74bec9 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:12:17.284281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.284828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.285253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.316871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:17.318111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:17.381372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1101/2167] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:17.390827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.392327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.392602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.413097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:17.414274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:17.423525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1067/2168] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:12:17.427539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.438703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829647-ea3e-4537-a56b-0ef8faa072a7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.443895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.445321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.445725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.445937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.446588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:17.446893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.448524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.464891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1102/2169] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:12:17.466045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.476143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0593f2-e643-431d-80fd-8388b55891ab None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.480982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.482837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.482837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.482837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.482837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.484807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.574950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1068/2170] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:17 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:17.576129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.586549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e388-ad6e-4653-9f55-855def94d2c6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.648506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.649232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.649590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.650088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.650748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.653547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.666715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:17.764836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1103/2171] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:17 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:17.766385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.776899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da69379-9f7a-435a-8473-007d29f5bfd1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.781821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.782425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.793785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.793785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.793785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.793785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.796436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1069/2172] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:17.800441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.811297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c524f5-c1aa-4550-825b-55fcff66cc41 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.815929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.816566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.816846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.817484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.817793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.819549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaec861-6fee-47a4-af2a-5164e2ff48bf admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.830944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1104/2173] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:17.835062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:17.845319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4634e-7cd9-40d9-b244-9f876c3a031e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:17.851320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:17.851320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.851320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.851693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] PATH_INFO: `/v3/projects/1a7a0e1ac76949879b727c9d57740482/users/5bcb811916d1403bbdedf6517ed5d3b0/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.867427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7a0e1ac76949879b727c9d57740482, user_id=5bcb811916d1403bbdedf6517ed5d3b0, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:17.870303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:17.939858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1070/2174] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:17 2022] PUT /identity/v3/projects/1a7a0e1ac76949879b727c9d57740482/users/5bcb811916d1403bbdedf6517ed5d3b0/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:17.951028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:17.951365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:17.951639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:17.979225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] MFA Rules not processed for user `5bcb811916d1403bbdedf6517ed5d3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:17.980411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:18.048036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1105/2175] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:18.053507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.118226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca00ddf-b080-4368-b23a-25f9200cd19e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.123051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.123680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.123969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.124238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.124904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.126881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.208905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1071/2176] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:18 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:18.214124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.224993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfe5b2-c6eb-4678-9f4c-64cb22fe01ae None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.280328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.280934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.281211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.281481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.282146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.284070 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.371619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1106/2177] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:18 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:18.372918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.383719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64c816-77c5-444a-ad74-2b80bcb8b768 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.388527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.389133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.389416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.389693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.390339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.391983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.404258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1072/2178] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:18.409231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.421722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbe8d5-18fc-495c-9c7c-cd1fe3696c7c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.428116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.431562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.431562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.431562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] PATH_INFO: `/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f/users/b4aa5fd4bea944f58904b88d1bdbbf19/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.447344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c16b10d8ba048bc9ec3056ddb6d161f, user_id=b4aa5fd4bea944f58904b88d1bdbbf19, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.450087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.513317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1107/2179] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:18 2022] PUT /identity/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f/users/b4aa5fd4bea944f58904b88d1bdbbf19/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:18.517538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.529965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bd17b-f185-4199-b4bf-7c76691df528 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.601917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.602631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.603032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.603402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.604117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.605799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c0cc0-1828-4017-b0ce-f451cfddac09 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.608540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:18.617479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1073/2180] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:18.622041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.633996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5eb53-edfd-4f85-8359-6154d331a039 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.639958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.640599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.640880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.648964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] PATH_INFO: `/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f/users/b4aa5fd4bea944f58904b88d1bdbbf19/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.648964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c16b10d8ba048bc9ec3056ddb6d161f, user_id=b4aa5fd4bea944f58904b88d1bdbbf19, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.648964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.667378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1108/2181] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:18 2022] PUT /identity/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f/users/b4aa5fd4bea944f58904b88d1bdbbf19/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:12:18.672869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.672869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.672869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.702117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] MFA Rules not processed for user `b4aa5fd4bea944f58904b88d1bdbbf19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:18.702988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:18.779857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1074/2182] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:18.791461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:18.793589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:18.834426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:18.880484 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0633a1-8a69-4417-a68d-e3ead697e0eb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:18.886442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:18.889303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:18.889303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:18.889303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:18.906207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:18.908898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fdfa5-be2e-442e-9ea0-53390d1486d0 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:18.920653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1109/2183] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:20.108263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:20.169746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958fe52-82b1-4773-9ba4-132a16c921af None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:20.174299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:20.174911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:20.175221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:20.175493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] PATH_INFO: `/v3/users/5bcb811916d1403bbdedf6517ed5d3b0` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:20.176593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcb811916d1403bbdedf6517ed5d3b0)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:20.178595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:20.393191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcb811916d1403bbdedf6517ed5d3b0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:20.405737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcb811916d1403bbdedf6517ed5d3b0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:20.447622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcb811916d1403bbdedf6517ed5d3b0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:20.448271 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcb811916d1403bbdedf6517ed5d3b0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:20.448887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcb811916d1403bbdedf6517ed5d3b0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:20.461334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1075/2184] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:20 2022] DELETE /identity/v3/users/5bcb811916d1403bbdedf6517ed5d3b0 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:20.465741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:20.466202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:20.466294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:20.485160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:20.486351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:20.544867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1110/2185] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:20.555497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:20.556858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:20.562179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:20.613781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2951f4-38fc-4c33-8ff6-4bfb993e9885 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:20.617976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:20.618591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:20.619119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:20.619441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:20.628892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:20.630321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:20.640380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1076/2186] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:21.112851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:21.114253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:21.129549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e608-398e-4f43-8f79-542edee81fea None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:21.135702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:21.136355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:21.136676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:21.137046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] PATH_INFO: `/v3/projects/1a7a0e1ac76949879b727c9d57740482` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:21.138061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7a0e1ac76949879b727c9d57740482)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:21.140095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:21.264939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7a0e1ac76949879b727c9d57740482', 'request_id': 'req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.367119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7a0e1ac76949879b727c9d57740482', 'request_id': 'req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.374025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7a0e1ac76949879b727c9d57740482', 'request_id': 'req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.384299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] The token cache is being invalidate because project 1a7a0e1ac76949879b727c9d57740482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:21.384582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.385219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.386688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1111/2187] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:21 2022] DELETE /identity/v3/projects/1a7a0e1ac76949879b727c9d57740482 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:21.390815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:21.401104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f6c45-8a77-4f05-b0cd-70e65b4ed3f1 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:21.461206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:21.461832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:21.462105 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:21.464490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] PATH_INFO: `/v3/users/b4aa5fd4bea944f58904b88d1bdbbf19` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:21.464490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4aa5fd4bea944f58904b88d1bdbbf19)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:21.465461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:21.639693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4aa5fd4bea944f58904b88d1bdbbf19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.652688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4aa5fd4bea944f58904b88d1bdbbf19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.722467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4aa5fd4bea944f58904b88d1bdbbf19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.723083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4aa5fd4bea944f58904b88d1bdbbf19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.723784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a2ceaa8b-d498-4737-9db2-70ed34696f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4aa5fd4bea944f58904b88d1bdbbf19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:21.736953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1077/2188] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:21 2022] DELETE /identity/v3/users/b4aa5fd4bea944f58904b88d1bdbbf19 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:12:22.205207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.206660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:22.223145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8a45-40a8-4e9c-8ef5-3a726208e42b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.272200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:22.281284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.281908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.282183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.282451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] PATH_INFO: `/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.283573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c16b10d8ba048bc9ec3056ddb6d161f)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.285469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:22.401900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c16b10d8ba048bc9ec3056ddb6d161f', 'request_id': 'req-b7e35593-4c90-4c28-9e03-62c8536d37ae'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:22.470886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c16b10d8ba048bc9ec3056ddb6d161f', 'request_id': 'req-b7e35593-4c90-4c28-9e03-62c8536d37ae'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:22.477918 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c16b10d8ba048bc9ec3056ddb6d161f', 'request_id': 'req-b7e35593-4c90-4c28-9e03-62c8536d37ae'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:22.487897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] The token cache is being invalidate because project 9c16b10d8ba048bc9ec3056ddb6d161f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:22.488180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:22.488807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b7e35593-4c90-4c28-9e03-62c8536d37ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:22.490276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1112/2189] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:22 2022] DELETE /identity/v3/projects/9c16b10d8ba048bc9ec3056ddb6d161f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:22.496244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.496747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.496855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.516521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:22.517720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:22.536043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:22.578136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1078/2190] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:22.587738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.588027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.588303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.608642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:22.609846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:22.617943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1113/2191] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:22.622202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.631580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262e4d-6c31-4dc5-9844-76d6771025b0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.638680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.638680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.638680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.638680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.639145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:22.639145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.639998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:22.652464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1079/2192] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:12:22.656445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.666159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83942e33-635d-4015-bb65-f855e6c57d36 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.671087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.671707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.671985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.672258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.672876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.674771 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:22.757941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1114/2193] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:22 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:22.762067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.772197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7971-d3a4-4002-8e37-8a60ec43d4a8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.827013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.827641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.827922 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.828193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.828811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.830727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:22.932774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1080/2194] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:22 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:22.936790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.947493 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3ad25-7f1e-4f35-84ad-e09e0f625b33 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.952309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.952898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.953178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.953449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.954275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.955920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:22.967016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1115/2195] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:22.970982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:22.981487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46fecc-5132-41a9-8553-46ddd3da2c75 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:22.986440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:22.987063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:22.987376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:22.987649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:22.988292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:22.989948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ee303-61f2-4600-9e2d-d09aa0d9c959 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:23.001601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1081/2196] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:23.005772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:23.016632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd46534-3efc-476c-9d30-8b47bf72429a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:23.021655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:23.022305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.022579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.022900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] PATH_INFO: `/v3/projects/f93d673737db4a2585fc426e30e57f60/users/e1c83388c7b845d693f1099f7a76eeba/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.040602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93d673737db4a2585fc426e30e57f60, user_id=e1c83388c7b845d693f1099f7a76eeba, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:23.044421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:23.127247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1116/2197] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:23 2022] PUT /identity/v3/projects/f93d673737db4a2585fc426e30e57f60/users/e1c83388c7b845d693f1099f7a76eeba/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:23.138316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.138600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.138874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.167315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] MFA Rules not processed for user `e1c83388c7b845d693f1099f7a76eeba`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:23.168762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:23.236038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1082/2198] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:23.251873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:23.251873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:23.304564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7050dbf-ef1e-4b83-b4a8-c00eac681896 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:23.309298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:23.309977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.310253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.310527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.319095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:23.320560 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503ff1d-ec2b-4b33-8b38-ddfb1af91fbc service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:23.329714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1117/2199] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:23.351766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:23.373381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:23.418168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebdd9f-b3f3-4817-81bd-2cc42a0fde95 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:23.422958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:23.423608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.423890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.424161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] PATH_INFO: `/v3/users/e1c83388c7b845d693f1099f7a76eeba` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.425294 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c83388c7b845d693f1099f7a76eeba)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:23.427288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:23.611642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c83388c7b845d693f1099f7a76eeba'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:23.624554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c83388c7b845d693f1099f7a76eeba'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:23.688118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c83388c7b845d693f1099f7a76eeba'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:23.688973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c83388c7b845d693f1099f7a76eeba'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:23.689658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c83388c7b845d693f1099f7a76eeba'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:23.702834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1083/2200] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:23 2022] DELETE /identity/v3/users/e1c83388c7b845d693f1099f7a76eeba => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:23.707351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.707640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.707914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.727658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:23.728848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:23.744869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:23.787090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1118/2201] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:23.797251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:23.798627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:23.854936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791735d-53ae-4edb-aceb-1a76d1864e65 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:23.858438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:23.859027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:23.859329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:23.859598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:23.868387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:23.869795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd614c8-3202-47f4-9182-baad3d69fdd6 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:23.878947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1084/2202] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:24.231702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.233144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:24.245579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3aece-6f7f-4459-a51d-3995b661cd47 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.250418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.251055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.251429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.251718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] PATH_INFO: `/v3/projects/f93d673737db4a2585fc426e30e57f60` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.252873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93d673737db4a2585fc426e30e57f60)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.254885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.367136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93d673737db4a2585fc426e30e57f60', 'request_id': 'req-57db79fa-8413-402a-bd03-2fd332655969'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:24.442657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93d673737db4a2585fc426e30e57f60', 'request_id': 'req-57db79fa-8413-402a-bd03-2fd332655969'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:24.450112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93d673737db4a2585fc426e30e57f60', 'request_id': 'req-57db79fa-8413-402a-bd03-2fd332655969'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:24.460537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] The token cache is being invalidate because project f93d673737db4a2585fc426e30e57f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:24.460812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:24.461500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-57db79fa-8413-402a-bd03-2fd332655969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:24.463002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1119/2203] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:24 2022] DELETE /identity/v3/projects/f93d673737db4a2585fc426e30e57f60 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:24.468743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.469081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.469462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.489715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:24.490897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:24.531915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:24.554175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1085/2204] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:24.563247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.563524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.563794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.584879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:24.586071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:24.594514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1120/2205] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:24.605316 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.614852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cec88-54d0-4d16-965d-e5eccb52654b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.621492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.622023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.622296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.622565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.623237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:24.623540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.625128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.636997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1086/2206] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:12:24.641615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.651296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46341c46-0e4e-4a70-9d33-fc362c438bfc None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.656068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.656660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.656935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.657202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.657830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.659833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.752138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1121/2207] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:24 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:24.756395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.766481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48046d-0671-4b49-99b3-27938b133e57 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.821386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.821967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.822244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.822515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.823164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.825062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.925719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1087/2208] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:24 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:24.928074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.939471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5be5-d716-42b1-bf6a-0ba8df013ba9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.944163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.944765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.945039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.945309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.945967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.947624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.957668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1122/2209] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:24.962213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:24.972067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e68ce-59de-4303-a315-05c81b30ce16 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:24.976582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:24.977856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:24.978133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:24.978399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:24.979030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:24.980641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba2bd0-edd3-48dd-99e4-d70b10535dc9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:24.990955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1088/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:24.995013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:25.004830 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02aa6a-40fd-461e-b2ac-38705c965bb4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:25.009541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:25.010185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:25.010461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:25.010761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] PATH_INFO: `/v3/projects/0074eec24f3848b8baa19ef37441a208/users/b99ec92c14334224bdba2b8dfe00a337/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:25.025756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0074eec24f3848b8baa19ef37441a208, user_id=b99ec92c14334224bdba2b8dfe00a337, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:25.028600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:25.127209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1123/2211] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:24 2022] PUT /identity/v3/projects/0074eec24f3848b8baa19ef37441a208/users/b99ec92c14334224bdba2b8dfe00a337/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:25.138366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:25.138678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:25.138951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:25.168560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] MFA Rules not processed for user `b99ec92c14334224bdba2b8dfe00a337`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:25.169736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:25.231400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1089/2212] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:25.242489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:25.244448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:25.289158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:25.300547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6282fea-2b42-4900-9e1f-4d502cb43bfa admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:25.304908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:25.305094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:25.305460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:25.305898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:25.314259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:25.315725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aba795-0dd4-48ac-a2c1-3bec577230dc service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:25.325402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1124/2213] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:35.228523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:35.356721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=7 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:44.054055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:44.055491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:44.120663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce54ae0-3280-4a66-a4e6-cf2839ed545a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:44.125560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:44.126411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:44.126847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:44.127283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] PATH_INFO: `/v3/users/a39c17cdfb204bea8af659548a2c476f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:44.128953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39c17cdfb204bea8af659548a2c476f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:44.131845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:44.343365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39c17cdfb204bea8af659548a2c476f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:44.355062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39c17cdfb204bea8af659548a2c476f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:44.420749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39c17cdfb204bea8af659548a2c476f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:44.421398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39c17cdfb204bea8af659548a2c476f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:44.421999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39c17cdfb204bea8af659548a2c476f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:44.436522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1090/2214] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:44 2022] DELETE /identity/v3/users/a39c17cdfb204bea8af659548a2c476f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:44.440998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:44.441488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:44.441589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:44.461283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:44.462474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:44.521555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1125/2215] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:44.531299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:44.532759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:44.592329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3577-cb43-4ce9-87dd-27fb259247d3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:44.595974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:44.596589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:44.596872 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:44.597149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:44.607136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:44.608581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd90bf0-5de6-4d12-9338-9d56b6d6ad51 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:44.617847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1091/2216] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:44.971542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:44.973030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:44.986544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8071-8ac2-4573-a965-a5a200bbbb31 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:44.991213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:44.991837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:44.992642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:44.992924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] PATH_INFO: `/v3/projects/0784cefbb9764465b4c9b36a4c5aecd7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:44.994212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0784cefbb9764465b4c9b36a4c5aecd7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:44.996157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.088984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0784cefbb9764465b4c9b36a4c5aecd7', 'request_id': 'req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:45.134991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0784cefbb9764465b4c9b36a4c5aecd7', 'request_id': 'req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:45.141708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0784cefbb9764465b4c9b36a4c5aecd7', 'request_id': 'req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:45.152072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] The token cache is being invalidate because project 0784cefbb9764465b4c9b36a4c5aecd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:12:45.152360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:45.153627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-08b20cf8-2d07-435d-b3c6-483cb4a9cdc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:12:45.155283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1126/2217] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:12:44 2022] DELETE /identity/v3/projects/0784cefbb9764465b4c9b36a4c5aecd7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:45.160211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.160673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.160763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.180771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:45.181897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:45.198380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:45.239758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1092/2218] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:45.247551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.247842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.248162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.268732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:45.269991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:45.278610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1127/2219] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:45.282879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.292363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a88e0-49aa-4d79-a869-140e2b4ed3ff None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.297855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.298427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.298708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.298983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.299659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:12:45.299967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.301568 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.313820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1093/2220] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:12:45.318073 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.327542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aed5ed-5468-4d18-a367-e4b651a829b2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.332289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.332982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.333257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.333522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.334141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.336085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.451078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1128/2221] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:12:45 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:45.455151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.465091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dae51b-b766-493e-9dfc-86b5f5e717f2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.522273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.522862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.523164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.523440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.524068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.525971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.615667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1094/2222] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:12:45 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:12:45.619936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.630418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c570c-d844-4e1f-af80-fa458241bb6d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.635059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.635672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.635949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.636225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.636897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.639306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.649490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1129/2223] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:45.654138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.664074 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc550a-9c3d-4289-be31-3596e45076ac admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.669113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.669748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.670046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.670311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.670998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.672778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86dace-05d3-4c9c-8c64-c834c33dfc50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.685392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1095/2224] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:12:45.689623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.699588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5f65d-20ec-4b6a-8c1d-7aae308576bb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.704977 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.705551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.705825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.706117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] PATH_INFO: `/v3/projects/1bbc69559c834081bae0961f7a0193aa/users/07e31de32b3c40efbe6404062b4ba88f/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.720805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbc69559c834081bae0961f7a0193aa, user_id=07e31de32b3c40efbe6404062b4ba88f, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.723479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.793398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1130/2225] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:12:45 2022] PUT /identity/v3/projects/1bbc69559c834081bae0961f7a0193aa/users/07e31de32b3c40efbe6404062b4ba88f/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:12:45.804913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46200755-8f81-4835-b05b-8a3f008c3fc8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.805208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46200755-8f81-4835-b05b-8a3f008c3fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.805487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46200755-8f81-4835-b05b-8a3f008c3fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.835205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-46200755-8f81-4835-b05b-8a3f008c3fc8 None None] MFA Rules not processed for user `07e31de32b3c40efbe6404062b4ba88f`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:45.836505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-46200755-8f81-4835-b05b-8a3f008c3fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:45.898086 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1096/2226] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:45.909269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:45.910902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:45.932345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:45.968104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d1810-705d-4823-a8e6-f733b3c6a75e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:45.971836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:45.972448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:45.972726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:45.972998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:45.982245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:45.986032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:12:45.992534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1131/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:12:55.508281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af140e-bfd4-4aac-b40d-4b611ccebf86 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:55.508792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af140e-bfd4-4aac-b40d-4b611ccebf86 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:55.508885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af140e-bfd4-4aac-b40d-4b611ccebf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:55.527992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-17af140e-bfd4-4aac-b40d-4b611ccebf86 None None] MFA Rules not processed for user `81816508df7c431fa3b4b495d11ac101`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:12:55.529182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-17af140e-bfd4-4aac-b40d-4b611ccebf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:55.541595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:55.586983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1097/2228] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:12:56.031376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=17 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:12:59.727556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:12:59.729025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:12:59.803629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3abbc-7c2c-435c-95ad-b2cc952290ae service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:12:59.808889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:12:59.809605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:12:59.813403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:12:59.813403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] PATH_INFO: `/v3/users/b99ec92c14334224bdba2b8dfe00a337` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:12:59.813403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99ec92c14334224bdba2b8dfe00a337)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:12:59.813885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:00.046000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99ec92c14334224bdba2b8dfe00a337'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.058955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99ec92c14334224bdba2b8dfe00a337'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.066922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99ec92c14334224bdba2b8dfe00a337'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.067758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99ec92c14334224bdba2b8dfe00a337'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.068459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99ec92c14334224bdba2b8dfe00a337'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.182614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1132/2229] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:12:59 2022] DELETE /identity/v3/users/b99ec92c14334224bdba2b8dfe00a337 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:00.187517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:00.188025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:00.188401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:00.208781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:00.209988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:00.275145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1098/2230] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:00.286036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:00.287762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:00.345730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1882-c31f-4719-97cd-383556ee8f1c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:00.349270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:00.349862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:00.350134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:00.350398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:00.360409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:00.362087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d1fbd-9b0c-4873-a82f-3c6b93c22e46 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:00.373505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1133/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:00.813513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:00.814363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:00.832627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efd87f-e394-4c47-be4d-ccc54702fb32 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:00.837723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:00.838359 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:00.838640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:00.842794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] PATH_INFO: `/v3/projects/0074eec24f3848b8baa19ef37441a208` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:00.842794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0074eec24f3848b8baa19ef37441a208)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:00.842794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:00.936368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0074eec24f3848b8baa19ef37441a208', 'request_id': 'req-071b350c-aa22-4edf-9328-99fd8024b9e7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.942699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0074eec24f3848b8baa19ef37441a208', 'request_id': 'req-071b350c-aa22-4edf-9328-99fd8024b9e7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:00.948893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0074eec24f3848b8baa19ef37441a208', 'request_id': 'req-071b350c-aa22-4edf-9328-99fd8024b9e7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:01.021502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] The token cache is being invalidate because project 0074eec24f3848b8baa19ef37441a208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:13:01.021796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:01.022414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-071b350c-aa22-4edf-9328-99fd8024b9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:01.023993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1099/2232] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:13:00 2022] DELETE /identity/v3/projects/0074eec24f3848b8baa19ef37441a208 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:01.028691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.029179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.029281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.049316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:01.050495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:01.073100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:01.115525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1134/2233] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:01.123325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.123613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.123892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.145089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:01.146284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:01.154121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1100/2234] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:01.158938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.168987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1a933-46eb-4078-b3aa-b908b95fc8c1 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.174120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.174710 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.174983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.175288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.175948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:13:01.176242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.177813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.190040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1135/2235] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:13:01.194416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.204443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9305b-aea0-4f68-91fb-befdf780459e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.209311 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.209897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.210175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.210445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.211067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.213006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.295466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1101/2236] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:13:01 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:13:01.297709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.308354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6853c-33dd-4e92-af72-40b2acb04d24 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.368852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.369460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.369867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.370146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.370778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.372767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.485173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1136/2237] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:13:01 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:13:01.489127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.499490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcd9ab-bcaa-4d1e-8a7b-76b61b5c4677 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.504901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.505525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.505758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.506030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.506668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.508304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.518941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1102/2238] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:13:01.522824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.533116 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69851d0d-3a3f-4680-9ab9-5ec1b07dec35 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.537820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.538401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.538674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.538941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.539613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.541295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2d93e-0cab-41a7-9110-437e7dcadb70 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.552301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1137/2239] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:13:01.556433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.567611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc50161-992b-47c4-bb39-c5bf3cf89e01 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.572278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.572950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.573230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.573500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] PATH_INFO: `/v3/projects/79d23226b59c44f7bd30d9386494f4b3/users/602d298896d643a098bcbc1853dddbb7/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.575940 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:01.589653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d23226b59c44f7bd30d9386494f4b3, user_id=602d298896d643a098bcbc1853dddbb7, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.592436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.649851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1103/2240] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:13:01 2022] PUT /identity/v3/projects/79d23226b59c44f7bd30d9386494f4b3/users/602d298896d643a098bcbc1853dddbb7/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:01.654994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad84-cedc-4e7d-935d-b72420db8817 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.655389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad84-cedc-4e7d-935d-b72420db8817 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.660529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad84-cedc-4e7d-935d-b72420db8817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.674511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-4166ad84-cedc-4e7d-935d-b72420db8817 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:01.675727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4166ad84-cedc-4e7d-935d-b72420db8817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:01.736607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1138/2241] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:13:01.746103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:01.747817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:01.804670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271df60-deb0-4070-b4d5-e3cc428621d2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:01.808341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:01.808937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:01.809220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:01.809493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:01.818874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:01.820333 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:01.829745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1104/2242] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:03.045480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1b0c-24cc-4948-99b4-2c0e79d60522 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:03.045998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1b0c-24cc-4948-99b4-2c0e79d60522 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:03.045998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1b0c-24cc-4948-99b4-2c0e79d60522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:03.075241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-135a1b0c-24cc-4948-99b4-2c0e79d60522 None None] MFA Rules not processed for user `602d298896d643a098bcbc1853dddbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:03.076362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-135a1b0c-24cc-4948-99b4-2c0e79d60522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:03.094725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:03.145009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1139/2243] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:03.156322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:03.159097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:03.218681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa131bb-ac5b-41c3-bedb-73858d9aa021 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:03.222456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:03.223054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:03.223363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:03.223638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:03.232112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:03.233552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:03.242573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1105/2244] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:05.956179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc16c4-ade8-4fc8-afc8-4b193a5956d1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:05.956844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc16c4-ade8-4fc8-afc8-4b193a5956d1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:05.956844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc16c4-ade8-4fc8-afc8-4b193a5956d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:05.975890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-80fc16c4-ade8-4fc8-afc8-4b193a5956d1 None None] MFA Rules not processed for user `602d298896d643a098bcbc1853dddbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:05.977157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-80fc16c4-ade8-4fc8-afc8-4b193a5956d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:05.984066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1140/2245] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:13.238569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:15.970224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=22 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:27.762946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:27.764368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:27.835868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354bcb2-2c2f-4eb1-90be-d0a0c6461011 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:27.841021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:27.841614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:27.841890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:27.842159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:27.843893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:27.855724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:27.857303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:27.866815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1106/2246] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:29.518251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fef0-292d-4ed0-95fb-866416552671 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:29.518725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fef0-292d-4ed0-95fb-866416552671 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:29.518822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fef0-292d-4ed0-95fb-866416552671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:29.538343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5070fef0-292d-4ed0-95fb-866416552671 None None] MFA Rules not processed for user `602d298896d643a098bcbc1853dddbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:29.539585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5070fef0-292d-4ed0-95fb-866416552671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:29.546572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1141/2247] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:37.862772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=32 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:39.532757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:45.913946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:45.915402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:45.928506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fdb7-66db-4f2b-88f2-d6f418e323be service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:45.932975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:45.933576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:45.933855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:45.934150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] PATH_INFO: `/v3/users/07e31de32b3c40efbe6404062b4ba88f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:45.935275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e31de32b3c40efbe6404062b4ba88f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:45.937182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:46.156675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e31de32b3c40efbe6404062b4ba88f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:46.168484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e31de32b3c40efbe6404062b4ba88f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:46.257943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e31de32b3c40efbe6404062b4ba88f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:46.270742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e31de32b3c40efbe6404062b4ba88f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:46.270742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e31de32b3c40efbe6404062b4ba88f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:46.273958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1107/2248] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:13:45 2022] DELETE /identity/v3/users/07e31de32b3c40efbe6404062b4ba88f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:46.278447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:46.282621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:46.282912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:46.302140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:46.303335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:46.362526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1142/2249] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:46.372385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:46.373863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:46.430731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba1fd-7fba-4e5e-970a-2f6ad7cdfad4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:46.434297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:46.434891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:46.435200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:46.435472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:46.445312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:46.450948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a6e4-5e81-467f-8754-5b3727563499 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:46.456936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1108/2250] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:46.939463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:46.940982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:46.952841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329a0c5-33f5-407c-878f-6fd66ff66ed0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:46.958062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:46.958627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:46.959909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:46.960204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] PATH_INFO: `/v3/projects/1bbc69559c834081bae0961f7a0193aa` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:46.961369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbc69559c834081bae0961f7a0193aa)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:46.963917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.058687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbc69559c834081bae0961f7a0193aa', 'request_id': 'req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:47.119965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbc69559c834081bae0961f7a0193aa', 'request_id': 'req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:47.127208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbc69559c834081bae0961f7a0193aa', 'request_id': 'req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:47.137507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] The token cache is being invalidate because project 1bbc69559c834081bae0961f7a0193aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:13:47.137794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:47.138420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e50ac294-6c3a-4b4c-9141-cb94a13f3d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:13:47.139936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1143/2251] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:13:46 2022] DELETE /identity/v3/projects/1bbc69559c834081bae0961f7a0193aa => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:47.145688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.146164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.146351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.165522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:47.166691 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:47.225523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1109/2252] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:47.233480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.233834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.234115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.254936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:47.256147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:47.264196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1144/2253] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:47.268599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.278252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029261e9-36b9-4045-986a-b0d31859e04f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.283133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.283722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.284003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.284283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.284999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:13:47.285308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.286906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.299699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1110/2254] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:13:47.309440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.319428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceecf12-06d7-4f80-82f4-0a72e4733849 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.324054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.324649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.324991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.325184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.325810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.327749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.459016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1145/2255] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:13:47 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:13:47.463444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.468664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:47.474017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5bab4-1e88-4e29-ae36-84d404cab673 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.529562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.530163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.530444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.530932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.531379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.533349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.629669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1111/2256] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:13:47 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:13:47.633575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.643902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fb26e-8cb8-4fc7-a814-a8b7cd1613af admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.648657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.649250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.649532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.649807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.650443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.652086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.662526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1146/2257] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:13:47.666418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.677498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca944e0-92ed-4327-a637-a02907d5814a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.682045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.682625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.682906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.683304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.683983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.685600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12780115-4c5e-497e-9eda-12c5642a7370 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.696571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1112/2258] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:13:47.700541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.710355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609da636-0810-4d83-ba9c-42cf0a546d58 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:47.714939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:47.715610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.715894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.716172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] PATH_INFO: `/v3/projects/4299d4d3bdaf4d97908a7e404f99d815/users/0f3c9be876d64ece967dc67974d600fa/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.731094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4299d4d3bdaf4d97908a7e404f99d815, user_id=0f3c9be876d64ece967dc67974d600fa, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:47.733783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:47.823124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1147/2259] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:13:47 2022] PUT /identity/v3/projects/4299d4d3bdaf4d97908a7e404f99d815/users/0f3c9be876d64ece967dc67974d600fa/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:13:47.890629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207a71d-b2f8-4bd7-a1f8-fba790a74782 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:47.891339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207a71d-b2f8-4bd7-a1f8-fba790a74782 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:47.891339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207a71d-b2f8-4bd7-a1f8-fba790a74782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:47.918480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7207a71d-b2f8-4bd7-a1f8-fba790a74782 None None] MFA Rules not processed for user `0f3c9be876d64ece967dc67974d600fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:47.919713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7207a71d-b2f8-4bd7-a1f8-fba790a74782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:47.984506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1113/2260] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:47.995296 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:13:47.997081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:48.020239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:13:48.056877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866131-3b65-4777-9e1f-5b276d6f1ed2 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:13:48.060476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:13:48.061220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:48.062910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:48.062910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:48.070475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:13:48.071959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44319bec-f63b-46a9-b3b2-974586f6c0dd service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:13:48.081304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1148/2261] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:13:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:13:50.759770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96d4d-dd81-4e38-a108-2496b3b50b35 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:50.760276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96d4d-dd81-4e38-a108-2496b3b50b35 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:50.760378 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96d4d-dd81-4e38-a108-2496b3b50b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:50.779046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f1b96d4d-dd81-4e38-a108-2496b3b50b35 None None] MFA Rules not processed for user `0f3c9be876d64ece967dc67974d600fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:50.780252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f1b96d4d-dd81-4e38-a108-2496b3b50b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:50.787545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1114/2262] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:13:55.071614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:13:55.072113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:13:55.072221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:13:55.095500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] MFA Rules not processed for user `02240dc931e140498d82c6d91efb60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:13:55.096861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:13:55.169672 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1149/2263] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:00.155535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c3bd-db5a-4b53-99d6-c0db25e26cd1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:00.155992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c3bd-db5a-4b53-99d6-c0db25e26cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:00.156081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c3bd-db5a-4b53-99d6-c0db25e26cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:00.175882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e963c3bd-db5a-4b53-99d6-c0db25e26cd1 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:00.177060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e963c3bd-db5a-4b53-99d6-c0db25e26cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:00.234253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1115/2264] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:00.277664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:00.279055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:00.350788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778cc8a-0395-42f1-87b3-b816913ae6a4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:00.355562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:00.356159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:00.356433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:00.356698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:00.365529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:00.367171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:00.375191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1150/2265] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:10.227588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:10.371935 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=74 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:17.432031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65dcfe-da97-4007-bfb1-e8214a7c810c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:17.432564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65dcfe-da97-4007-bfb1-e8214a7c810c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:17.432670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65dcfe-da97-4007-bfb1-e8214a7c810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:17.452776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3b65dcfe-da97-4007-bfb1-e8214a7c810c None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:17.454026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3b65dcfe-da97-4007-bfb1-e8214a7c810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:17.461680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1116/2266] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:17.474190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:17.484569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21507a-b62a-415f-bbaa-b68539dd1753 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:17.489890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:17.490480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:17.490760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:17.491033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:17.499608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:17.501195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:17.509984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1151/2267] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:17.868080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691dd94-7595-4a2a-a55b-87dd4c17c16e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:17.868646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691dd94-7595-4a2a-a55b-87dd4c17c16e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:17.868952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691dd94-7595-4a2a-a55b-87dd4c17c16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:17.888455 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1691dd94-7595-4a2a-a55b-87dd4c17c16e None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:17.889685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1691dd94-7595-4a2a-a55b-87dd4c17c16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:17.897416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1117/2268] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:14:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:17.910108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:17.920576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccdb9e-d271-4b81-b178-8017f8c73414 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:17.926522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:17.927153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:17.927430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:17.927695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:17.935498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:17.937050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:17.945846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1152/2269] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 09:14:18.368352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576fde-48e1-4987-b641-74e0177a0acf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:18.368638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576fde-48e1-4987-b641-74e0177a0acf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:18.368914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1576fde-48e1-4987-b641-74e0177a0acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:18.398915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b1576fde-48e1-4987-b641-74e0177a0acf None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:18.400181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b1576fde-48e1-4987-b641-74e0177a0acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:18.408557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1118/2270] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:14:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:18.424436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:18.435339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a6888-9df7-464e-b693-e48b76375999 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:18.441179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:18.441872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:18.442207 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:18.442549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:18.451163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:18.452817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:18.461748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1153/2271] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:19.126745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066061-1be4-4814-b38a-05a4619d07dd None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:19.127772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066061-1be4-4814-b38a-05a4619d07dd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:19.128758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066061-1be4-4814-b38a-05a4619d07dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:19.149001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-91066061-1be4-4814-b38a-05a4619d07dd None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:19.150161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-91066061-1be4-4814-b38a-05a4619d07dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:19.157663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1119/2272] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:19.169499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:19.180383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85f39b-53ff-45c9-b8e2-155535fe7ef0 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:19.185937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:19.189144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:19.189144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:19.189144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:19.195802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:19.197374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:19.205996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1154/2273] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:19.889419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:19.889712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:19.889989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:19.912523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:19.913761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:19.922507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1120/2274] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:14:19.936979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:19.950769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567f05d-dd74-42c6-b8e6-6a9cf723d2a9 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:19.960067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:19.960681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:19.960959 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:19.961228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:19.973920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:19.977205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:19.987408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1155/2275] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:21.447883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:21.449377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:21.531887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b072315-94e7-46fa-8a78-706fc2f87759 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:21.537217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:21.537844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:21.538116 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:21.538383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] PATH_INFO: `/v3/users/602d298896d643a098bcbc1853dddbb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:21.539531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=602d298896d643a098bcbc1853dddbb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:21.541445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:21.792007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602d298896d643a098bcbc1853dddbb7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:21.820661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602d298896d643a098bcbc1853dddbb7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:21.891464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602d298896d643a098bcbc1853dddbb7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:21.892273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602d298896d643a098bcbc1853dddbb7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:21.892978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab00e729-5be4-4675-968d-3abb85fcb7aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602d298896d643a098bcbc1853dddbb7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:21.906603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1121/2276] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:14:21 2022] DELETE /identity/v3/users/602d298896d643a098bcbc1853dddbb7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:14:22.166054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.178714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503504c-8cb6-4022-a726-2dd8f7acccc9 service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.255506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.256183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.256430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.256702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] PATH_INFO: `/v3/projects/79d23226b59c44f7bd30d9386494f4b3` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.257831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d23226b59c44f7bd30d9386494f4b3)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.259874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.379684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d23226b59c44f7bd30d9386494f4b3', 'request_id': 'req-07f8947a-d009-4e5d-90c5-79bc384eafa2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:22.480706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d23226b59c44f7bd30d9386494f4b3', 'request_id': 'req-07f8947a-d009-4e5d-90c5-79bc384eafa2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:22.488825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d23226b59c44f7bd30d9386494f4b3', 'request_id': 'req-07f8947a-d009-4e5d-90c5-79bc384eafa2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:22.502151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] The token cache is being invalidate because project 79d23226b59c44f7bd30d9386494f4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:14:22.502474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:22.503150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-07f8947a-d009-4e5d-90c5-79bc384eafa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:22.504660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1156/2277] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:14:22 2022] DELETE /identity/v3/projects/79d23226b59c44f7bd30d9386494f4b3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:14:22.509574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.510069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.510171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.530026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:22.531249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:22.575955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:22.593025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1122/2278] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:22.600676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.600955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.601226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.621866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:22.623064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:22.631696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1157/2279] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:22.635983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.645496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbe3c-13a2-4782-a232-d16531d65d2b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.649976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.654828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.665013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1123/2280] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:14:22.669271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.679598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb3ec0-5e9c-4ca1-acfb-c4fc860cd29a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.684347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.684938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.685216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.685489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.686113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.688049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.704641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:22.779799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1158/2281] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:14:22 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:14:22.784986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.795014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff2e8-acf3-446e-ac41-887ceff147e7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.844351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.846285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:22.854244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.854874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.859354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.860527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.861204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.863265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.909992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac42e2-0d15-4c4a-9913-97b01337e7dc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.914619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.915264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.915561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.915809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] PATH_INFO: `/v3/users/0f3c9be876d64ece967dc67974d600fa` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.916920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f3c9be876d64ece967dc67974d600fa)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.918831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.955551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1124/2282] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:14:22 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:14:22.959709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:22.970662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ed32d-b40f-4568-881c-98c492213e09 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:22.976142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:22.976821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:22.977138 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:22.977492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:22.978202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:22.980060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:22.993699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1125/2283] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:14:22.997491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:23.008149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d5375-34fb-4141-b2d8-5f51e23ce64c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:23.012993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:23.013604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.013871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.014143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.014807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:23.017109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:23.027254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1126/2284] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:14:23.032696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:23.043535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b29fc-66a0-4954-9a3b-48c70074594b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:23.048782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:23.049437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.049729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.049996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] PATH_INFO: `/v3/projects/14201dd9cdf946d2824c31e55c0573f8/users/e64f94da55234c21889ecf1cd2220ed5/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.066006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14201dd9cdf946d2824c31e55c0573f8, user_id=e64f94da55234c21889ecf1cd2220ed5, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:23.068924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f60ca-ae04-499b-8a46-d359260da769 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:23.114315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f3c9be876d64ece967dc67974d600fa'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.129882 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f3c9be876d64ece967dc67974d600fa'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.137411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f3c9be876d64ece967dc67974d600fa'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.138281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f3c9be876d64ece967dc67974d600fa'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.138921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f3c9be876d64ece967dc67974d600fa'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.160989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1127/2285] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:14:23 2022] PUT /identity/v3/projects/14201dd9cdf946d2824c31e55c0573f8/users/e64f94da55234c21889ecf1cd2220ed5/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:14:23.165505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603d16-e269-44f5-b8a0-37791090a72d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.166389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603d16-e269-44f5-b8a0-37791090a72d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.166389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603d16-e269-44f5-b8a0-37791090a72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.189190 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a4603d16-e269-44f5-b8a0-37791090a72d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:23.190871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a4603d16-e269-44f5-b8a0-37791090a72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:23.252065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1128/2286] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:14:23.252408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1159/2287] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:14:22 2022] DELETE /identity/v3/users/0f3c9be876d64ece967dc67974d600fa => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:14:23.258705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.258989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.259363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.270007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:23.271250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:23.280140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:23.281319 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:23.296694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:23.339594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d715e-86ce-47f2-a38a-f8e3dc948ec8 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:23.343694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:23.344405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.344676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.345093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.363091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1129/2288] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:23.375582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:23.383646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:23.385944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5066e-38dd-4b4f-ba66-5d2d9c5343fd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:23.392115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:23.396466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.396466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.396466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.403929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:23.405679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:23.417737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1130/2289] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:14:23.425346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:23.426988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965897da-90f9-4dcd-a864-fa79b271440f service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:23.439231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1160/2290] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:14:23.721014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:23.722961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:23.746093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bb36-34d6-4904-8dd7-6581cecba29d service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:23.762969 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:23.763716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:23.765064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:23.765064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] PATH_INFO: `/v3/projects/4299d4d3bdaf4d97908a7e404f99d815` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:23.765780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4299d4d3bdaf4d97908a7e404f99d815)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:23.768194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:23.875851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4299d4d3bdaf4d97908a7e404f99d815', 'request_id': 'req-a1ca3985-275f-4a72-8f67-b78e73e0aca1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.882905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4299d4d3bdaf4d97908a7e404f99d815', 'request_id': 'req-a1ca3985-275f-4a72-8f67-b78e73e0aca1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.890025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4299d4d3bdaf4d97908a7e404f99d815', 'request_id': 'req-a1ca3985-275f-4a72-8f67-b78e73e0aca1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.965323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] The token cache is being invalidate because project 4299d4d3bdaf4d97908a7e404f99d815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:14:23.965710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.966434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:14:23.968195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1131/2291] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:14:23 2022] DELETE /identity/v3/projects/4299d4d3bdaf4d97908a7e404f99d815 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:14:24.201197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9826fd-7fa8-4a15-999f-c7003729922e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:24.201564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9826fd-7fa8-4a15-999f-c7003729922e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:24.201903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9826fd-7fa8-4a15-999f-c7003729922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:24.225643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ba9826fd-7fa8-4a15-999f-c7003729922e None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:24.227245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ba9826fd-7fa8-4a15-999f-c7003729922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:24.291623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1161/2292] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:24.307024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:24.307372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:24.387404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca3985-275f-4a72-8f67-b78e73e0aca1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:24.393038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:24.393685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:24.393957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:24.394223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:24.403358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:24.405130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:24.414717 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1132/2293] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:31.128835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:31.129326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:31.129425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:31.174548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] MFA Rules not processed for user `e64f94da55234c21889ecf1cd2220ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:31.175955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:31.312343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1162/2294] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:31.316210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:31.318397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:31.353058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:31.446367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e346c-abb6-4d1d-93a5-729ed3ce4089 service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:31.453875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:31.454628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:31.455011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:31.456720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:31.457535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:31.461812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:31.560682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1133/2295] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:14:31 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:14:31.562004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:31.581055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa471d-bbbb-4696-825b-5dc02c75d5b7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:31.681982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:31.682597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:31.682880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:31.683224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:31.684919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:31.686841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:31.699169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:31.807992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1163/2296] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:14:31 2022] POST /identity/v3/users => generated 299 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:14:31.810759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:31.823166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ccafd-8ad4-4ad9-baf1-87317670fdf9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:31.829585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:31.830396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:31.830870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:31.831237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:31.831895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:31.836482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:31.846032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1134/2297] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:14:31.850097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:31.864013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded74cff-c680-4b9c-b584-fd423bf62e90 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:31.869688 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:31.870350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:31.870626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:31.870902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] PATH_INFO: `/v3/projects/1ef6904564574c4dbfc996fb74e05751/users/df0f6118586342a18d14a4f071025933/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:31.898885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef6904564574c4dbfc996fb74e05751, user_id=df0f6118586342a18d14a4f071025933, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:31.902072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:31.988732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1164/2298] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:14:31 2022] PUT /identity/v3/projects/1ef6904564574c4dbfc996fb74e05751/users/df0f6118586342a18d14a4f071025933/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:14:31.993016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:32.005495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2bc0d-8edc-4562-8be3-af1211de604a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:32.087520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:32.088057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:32.088324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:32.088600 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:32.089239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:32.090881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b322-e51a-4b12-8e70-e7768cb65fcf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:32.105878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1135/2299] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:14:32.108088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:32.120801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056af4-5553-44f7-981e-fcee7d2fdad6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:32.128740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:32.129741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:32.130368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:32.130969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] PATH_INFO: `/v3/projects/1ef6904564574c4dbfc996fb74e05751/users/df0f6118586342a18d14a4f071025933/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:32.135948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef6904564574c4dbfc996fb74e05751, user_id=df0f6118586342a18d14a4f071025933, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:32.139757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:32.151621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1165/2300] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:14:32 2022] PUT /identity/v3/projects/1ef6904564574c4dbfc996fb74e05751/users/df0f6118586342a18d14a4f071025933/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:14:40.128894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f37d-2f2e-4177-a0bf-e1bec757d71d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:40.129439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f37d-2f2e-4177-a0bf-e1bec757d71d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:40.129439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f37d-2f2e-4177-a0bf-e1bec757d71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:40.169519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f115f37d-2f2e-4177-a0bf-e1bec757d71d None None] MFA Rules not processed for user `df0f6118586342a18d14a4f071025933`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:40.171247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f115f37d-2f2e-4177-a0bf-e1bec757d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:40.254343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1136/2301] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:40.267347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:40.269246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:40.327817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55620-045d-4905-8f4f-9b5ba61784b1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:40.331621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:40.332234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:40.332514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:40.332789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:40.390660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:40.392132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:40.401428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1166/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:14:40.673411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242acb8-66b0-41a4-a97b-a670b8492a90 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:40.673838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242acb8-66b0-41a4-a97b-a670b8492a90 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:40.673951 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242acb8-66b0-41a4-a97b-a670b8492a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:40.693681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b242acb8-66b0-41a4-a97b-a670b8492a90 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:40.694867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b242acb8-66b0-41a4-a97b-a670b8492a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:40.704160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:40.750997 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1137/2303] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:40.763669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:40.829954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d931c4-3a98-4408-805f-c655fd6d9058 service glance] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:40.834839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:40.835461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:40.835732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:40.835998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:40.842353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:14:40.844357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:40.845936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5de80e-fc2e-4ee0-b0a1-0178f9b79769 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:40.854030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1167/2304] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:46.829150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dab73-d2c2-4ed9-a490-9ca32d966f72 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:46.830309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dab73-d2c2-4ed9-a490-9ca32d966f72 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:46.830309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dab73-d2c2-4ed9-a490-9ca32d966f72 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:46.831638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1138/2305] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:14:46.839817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136fcf-8b9e-4f9a-981b-4eec3b881145 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:46.839817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136fcf-8b9e-4f9a-981b-4eec3b881145 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:46.840516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07136fcf-8b9e-4f9a-981b-4eec3b881145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:46.865844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-07136fcf-8b9e-4f9a-981b-4eec3b881145 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:46.867350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-07136fcf-8b9e-4f9a-981b-4eec3b881145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:46.988609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1168/2306] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:46.997933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dab73-d2c2-4ed9-a490-9ca32d966f72 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:47.010347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dab73-d2c2-4ed9-a490-9ca32d966f72 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:47.014834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:47.015483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.015765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.016035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.028574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:47.030078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea0fc7-c5e4-4168-a61a-2d1aa3312f87 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:47.044249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1139/2307] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:14:47.271921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.271921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.271921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.274444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1169/2308] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:14:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:14:47.281263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780db65-1e91-4780-be5d-797dd371aed7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.281602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780db65-1e91-4780-be5d-797dd371aed7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.281889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780db65-1e91-4780-be5d-797dd371aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.304591 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8780db65-1e91-4780-be5d-797dd371aed7 None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:47.333119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8780db65-1e91-4780-be5d-797dd371aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:47.391383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1140/2309] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:14:47.404433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:47.411335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:47.428904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ee17a-0a73-4958-9ba4-ab75d09d4571 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:47.432669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:47.433439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.433727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.434004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.443552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:47.445679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236708a9-62ea-4d8a-be6a-c79305b49736 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:47.456803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1170/2310] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:14:47.595186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.595186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.595186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.595186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1141/2311] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:14:47.596054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5064-a823-4fba-9f8f-0aae19eb0518 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.596054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5064-a823-4fba-9f8f-0aae19eb0518 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.596054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5064-a823-4fba-9f8f-0aae19eb0518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.612454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-120a5064-a823-4fba-9f8f-0aae19eb0518 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:47.613655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-120a5064-a823-4fba-9f8f-0aae19eb0518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:47.624272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1171/2312] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:14:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:47.635919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:47.639993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:47.654275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208c57e-dceb-4a31-821d-6ece7e7ab381 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:47.657931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:47.658522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:47.658801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:47.659070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:47.670084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:47.671569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:47.682986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1142/2313] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:14:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:14:51.831373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daed49-d1ad-4701-95c1-270a1e6310c6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:51.831866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daed49-d1ad-4701-95c1-270a1e6310c6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:51.831975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daed49-d1ad-4701-95c1-270a1e6310c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:51.853152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f0daed49-d1ad-4701-95c1-270a1e6310c6 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:51.854388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f0daed49-d1ad-4701-95c1-270a1e6310c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:51.861893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1172/2314] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:14:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:14:51.874380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:14:51.886213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395f943-6712-4c76-9b70-4b9f07bd36a7 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:14:51.891239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:14:51.891849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:51.892124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:51.892392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:51.901227 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:14:51.903445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c3975-266b-4000-8ca5-4bad3f82ed67 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:14:51.910973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1143/2315] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:14:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:14:52.091010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:14:52.091379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:14:52.091657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:14:52.113408 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] MFA Rules not processed for user `e64f94da55234c21889ecf1cd2220ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:14:52.114852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:14:52.123691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1173/2316] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:14:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:15:01.907709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=60 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:02.106900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=48 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:09.204047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:09.207514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:09.207514 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:09.225569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:15:09.230496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:09.233957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1144/2317] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:15:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:15:09.258191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:09.270368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622aa64-9ba4-445a-a119-afe0e2163cec None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:09.275315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:09.275905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:09.276177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:09.276444 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:09.285427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:09.287111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:09.295475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1174/2318] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:15:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 09:15:09.541420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:09.554014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee0da-24c0-41ba-8f3e-4f23fb953135 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:09.570876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:09.572332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:09.572332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:09.575128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:09.582438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:09.619696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:09.630924 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:09.633142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:09.635667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2620696-6f7e-4978-9c54-d71adf4c1c0f service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:09.639860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:09.641585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:09.645415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:09.645415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:09.647574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:09.672721 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1145/2319] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:09.680589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:09.683030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:09.731362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1175/2320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:19.665354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=9 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:19.691292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=12 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:28.664915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:28.674366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bb6b1-9067-41af-a89e-89212f87b56c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:28.679085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:28.679799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:28.680155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:28.680522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:28.682976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:28.749021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:28.750456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31612508-f13f-41cd-896f-a34d37aaffee service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:28.767577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1146/2321] 10.222.0.46 () {60 vars in 1396 bytes} [Thu Sep 1 09:15:28 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:32.384720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:32.386127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:32.400593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb59ac-31d8-4eea-9670-b1df80987e8f service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:32.406109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:32.406695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:32.406975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:32.407277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:32.408924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:32.466170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:32.467800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:32.476702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1176/2322] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:33.561269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:33.561759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:33.561855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:33.582689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:15:33.583936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:33.591722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1147/2323] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:15:33.603732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:33.615878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27dfc2-941b-4fd3-9baa-a240734c7f05 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:33.621137 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:33.621767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:33.622039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:33.622311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:33.630546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:33.632340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc789937-ef15-4628-af3e-c871e90ff559 service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:33.640405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1177/2324] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:15:35.440096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:35.449491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f259d-d312-4122-80ca-b63e681928cd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:35.453061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:35.453772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:35.454051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:35.454323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:35.456137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:35.466110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:35.467626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:35.475415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1148/2325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:43.637016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=34 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:45.472468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=37 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:15:48.789711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:48.790292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:48.790292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:48.810681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] MFA Rules not processed for user `81816508df7c431fa3b4b495d11ac101`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:15:48.811849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:48.818681 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1178/2326] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:15:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:15:51.095446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:51.104467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18bf4-a688-42b3-b83b-886c96c12a05 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:51.108043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:51.108619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:51.108894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:51.109163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:51.117842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:51.120022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:51.129187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1149/2327] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:53.330580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:53.348238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0c584-eb78-413e-aa77-c7b28373ab9e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:53.354258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:53.354892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:53.355197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:53.355501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:53.357219 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:53.370687 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:53.372405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:53.382890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1179/2328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:56.569989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:56.571907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:56.591798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f477f-95ad-4cbc-902c-8816c52a3c0c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:56.599047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:56.599888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:56.600238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:56.600582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:56.603409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:56.620403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:56.622196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:56.635031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1150/2329] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:15:58.120342 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:15:58.131611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c47d-9616-485d-8165-fcaa72b7a87c service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:15:58.137250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:15:58.137851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:15:58.138215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:15:58.138487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:15:58.140597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:15:58.203507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:15:58.205197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cfbfb-bea2-4572-85a7-9a0c85f55e0e service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:15:58.214872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1180/2330] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:16:06.008815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:16:06.019925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fb19c-b5c5-48b0-98ff-d0ace6a70320 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:16:06.025499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:16:06.026087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:06.026367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:06.026640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:06.028350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:06.085122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:16:06.086705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:16:06.095501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1151/2331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:16:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:16:08.209851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=37 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:16:16.091423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=40 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:16:21.971861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:21.972380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:21.972380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:21.991792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] MFA Rules not processed for user `02240dc931e140498d82c6d91efb60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:16:21.992974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:21.999955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1181/2332] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:16:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:16:22.298866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:16:22.308355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950f38f-6b1e-49da-94b5-fdf16e59b5f0 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:16:22.311900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:16:22.312481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:22.312754 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:22.313025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:22.315260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:22.324540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:16:22.325945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:16:22.334761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1152/2333] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:16:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:16:22.727082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:16:22.728645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:22.801250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5476b3-40b8-4d15-9dcf-96f99417b2b1 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:16:22.806236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:16:22.806835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:22.807122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:22.807410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:22.809113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:22.821562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:16:22.823227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf98735-ecbd-43d5-9336-14c57263e705 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:16:22.833136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1182/2334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:16:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:16:32.331344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:16:32.828930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=35 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:16:33.324541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:16:33.340203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1d31c-1709-4db3-9e6e-a03d0f2b05a0 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:16:33.345746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:16:33.346343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:33.346624 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:33.346895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:33.348699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:33.362370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:16:33.364094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:16:33.377677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1153/2335] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:16:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:16:43.373620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:16:52.741021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:16:52.741787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:16:52.741787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:16:52.760958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] MFA Rules not processed for user `3d931a379404433b9c3c690dd9beb83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:16:52.762153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:16:52.769559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1183/2336] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:16:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:02.755586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:27.784353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:27.794486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4b152-f744-4e13-8467-33761c4fcd92 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:27.798542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:27.799383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:27.799672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:27.799970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:27.801802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:27.871410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:27.872943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:27.885421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1154/2337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:31.967159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:31.968814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:31.984828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fccc0-879b-421a-b221-aeefcdaa26ff None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:31.993446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:31.994315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:31.994592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:31.994859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:31.996621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:32.013275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:32.014890 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc51008-fe48-48e4-819f-136355934227 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:32.026339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1184/2338] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:32.105958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:32.107497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:32.123629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c56e4-e31a-4f96-9e08-a0939ba67a8b service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:32.128304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:32.128928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:32.129584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:32.129584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] PATH_INFO: `/v3/users/81816508df7c431fa3b4b495d11ac101` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:32.130737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=81816508df7c431fa3b4b495d11ac101)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:32.133616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:32.350837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81816508df7c431fa3b4b495d11ac101'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.362643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81816508df7c431fa3b4b495d11ac101'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.446407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81816508df7c431fa3b4b495d11ac101'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.446980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81816508df7c431fa3b4b495d11ac101'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.447640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81816508df7c431fa3b4b495d11ac101'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.461793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1155/2339] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:17:32 2022] DELETE /identity/v3/users/81816508df7c431fa3b4b495d11ac101 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:32.506552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb189d-88d3-4e48-a0b0-bd452490fdcc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:32.507403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb189d-88d3-4e48-a0b0-bd452490fdcc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:32.507689 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb189d-88d3-4e48-a0b0-bd452490fdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:32.526686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d3bb189d-88d3-4e48-a0b0-bd452490fdcc None None] MFA Rules not processed for user `004ab0c0b138457e9a5e9288d039b932`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:32.527930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d3bb189d-88d3-4e48-a0b0-bd452490fdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:32.594848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1185/2340] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:32.710261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:32.711665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:32.786458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7e007-fcdd-4771-b747-d2fcb0c83eca admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:32.792295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:32.793033 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:32.793556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:32.793910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] PATH_INFO: `/v3/projects/15f1ee0d68f646bcad60cbf31c3c684d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:32.796062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f1ee0d68f646bcad60cbf31c3c684d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:32.798375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:32.809072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:32.908516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f1ee0d68f646bcad60cbf31c3c684d', 'request_id': 'req-ed99ea14-850f-45c7-8b85-cfa8030358ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.914578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f1ee0d68f646bcad60cbf31c3c684d', 'request_id': 'req-ed99ea14-850f-45c7-8b85-cfa8030358ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:32.922104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f1ee0d68f646bcad60cbf31c3c684d', 'request_id': 'req-ed99ea14-850f-45c7-8b85-cfa8030358ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:33.010747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] The token cache is being invalidate because project 15f1ee0d68f646bcad60cbf31c3c684d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:17:33.011041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:33.011758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed99ea14-850f-45c7-8b85-cfa8030358ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:33.013356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1156/2341] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:17:32 2022] DELETE /identity/v3/projects/15f1ee0d68f646bcad60cbf31c3c684d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:33.018101 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.018615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.018743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.042006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:33.043489 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:33.104591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1186/2342] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:17:33.112272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.112765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.112859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.133370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:33.134579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:33.143080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1157/2343] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:17:33.148185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.158287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fa615-df7e-4bdf-9713-a1f80b09657d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.163666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.164272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.164549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.164819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.165462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:17:33.165781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.167498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.179685 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1187/2344] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:17:33.184106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.193335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8405630-ea11-49c7-9194-4271f6fba8de None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.198032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.198618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.198892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.201151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.201151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.201716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.323833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1158/2345] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:17:33 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:17:33.327001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.339591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e4cd0-3c40-48aa-b654-809e4db89654 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.419298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.419908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.420184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.420452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.421085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.423006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.510154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1188/2346] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:17:33 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:17:33.514161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.525478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e074-bbcc-4928-adeb-834d54bceb9e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.530094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.530680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.530959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.531276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.531915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.533511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.543595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1159/2347] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:17:33.547593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.558145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1b71b-40ab-422c-ab6b-75b83c5ab60d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.562993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.563645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.563925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.564199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.564867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.566495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020cff5-28b1-45bb-b21b-9439ee9a27d0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.569247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:33.578241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1189/2348] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:17:33.582599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.593499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642ba10-1ca2-486d-a5c4-9495e78665fa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.605503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.605503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.605503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.605503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] PATH_INFO: `/v3/projects/feeebade381c4f288e48ad8ebf94084f/users/d89cd06dd6924910a9d8b56731cfb368/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.614124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=feeebade381c4f288e48ad8ebf94084f, user_id=d89cd06dd6924910a9d8b56731cfb368, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.617002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.710436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1160/2349] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:17:33 2022] PUT /identity/v3/projects/feeebade381c4f288e48ad8ebf94084f/users/d89cd06dd6924910a9d8b56731cfb368/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:33.721837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.722125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.722403 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.752278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] MFA Rules not processed for user `d89cd06dd6924910a9d8b56731cfb368`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:33.753547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:33.820200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1190/2350] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:33.831203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:33.832953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:33.912249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2eca2-30b2-43b0-b59d-caf44b5421fc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:33.918263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:33.918892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:33.919203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:33.919475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:33.927209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:33.929374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8d92c-0815-4024-a37d-f9b479c678be service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:33.938301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1161/2351] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:43.816623 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=33 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:43.934057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:44.114980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:44.180709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d679b-8581-41ba-83e0-9c7caf04a162 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:44.185645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:44.186264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:44.186539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:44.186807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] PATH_INFO: `/v3/users/d89cd06dd6924910a9d8b56731cfb368` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:44.187934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89cd06dd6924910a9d8b56731cfb368)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:44.190081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:44.359664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89cd06dd6924910a9d8b56731cfb368'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:44.372120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89cd06dd6924910a9d8b56731cfb368'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:44.378799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89cd06dd6924910a9d8b56731cfb368'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:44.379485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89cd06dd6924910a9d8b56731cfb368'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:44.380098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89cd06dd6924910a9d8b56731cfb368'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:44.483213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1191/2352] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:17:44 2022] DELETE /identity/v3/users/d89cd06dd6924910a9d8b56731cfb368 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:44.487696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:44.487982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:44.488255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:44.508556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:44.509744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:44.567747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1162/2353] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:44.577955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:44.579402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:44.639092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4c793-2a6a-4b8e-a2f0-641d5bf17be3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:44.642620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:44.643252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:44.643526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:44.643791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:44.652953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:44.654380 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ce757-3df3-4b5f-b9f9-f478478c4004 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:44.664858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1192/2354] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:45.084298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.088788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:45.098460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68ca60-4fcc-4288-8deb-3c1b6d68a150 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.105679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.106111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.106374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.106637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] PATH_INFO: `/v3/projects/feeebade381c4f288e48ad8ebf94084f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.107810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] RBAC: Authorizing `identity:delete_project(project_id=feeebade381c4f288e48ad8ebf94084f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.110352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.215024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feeebade381c4f288e48ad8ebf94084f', 'request_id': 'req-6fc39be7-8938-4feb-8135-7e823d79022c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:45.220457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feeebade381c4f288e48ad8ebf94084f', 'request_id': 'req-6fc39be7-8938-4feb-8135-7e823d79022c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:45.228557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feeebade381c4f288e48ad8ebf94084f', 'request_id': 'req-6fc39be7-8938-4feb-8135-7e823d79022c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:45.338948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] The token cache is being invalidate because project feeebade381c4f288e48ad8ebf94084f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:17:45.339336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:45.339954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6fc39be7-8938-4feb-8135-7e823d79022c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:17:45.341439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1163/2355] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:17:45 2022] DELETE /identity/v3/projects/feeebade381c4f288e48ad8ebf94084f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:45.346361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.346863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.346958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.369474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:45.370702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:45.388842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:45.432910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1193/2356] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:45.440868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.441295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.441802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.462683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:45.463953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:45.472179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1164/2357] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:45.477053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.486850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae524ba4-7e42-49c6-8d89-5c4af5078380 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.492641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.493274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.493549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.493817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.494454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:17:45.494754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.496429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.509002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1194/2358] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:17:45.513396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.522882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4630af-ee4e-43a8-90bc-2d2c672c1902 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.527674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.528268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.528550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.528825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.529464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.531488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.623131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1165/2359] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:17:45 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:17:45.627385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.637770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddc11-56c2-487a-811f-c7303da1d0e5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.692875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.693465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.693810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.694135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.694745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.696694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.785779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1195/2360] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:17:45 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:17:45.790637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.801078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecafbb1-8a06-4e70-81db-ce120c0085a3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.805756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.806384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.806666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.806941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.807652 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.809269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.819324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1166/2361] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:17:45.823333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.833338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469cc43-2fab-4685-8a5b-082b8aecd896 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.838810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.839433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.839713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.839984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.840650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.842241 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfebc-3fd2-45f1-9c59-db852f5834f4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.853239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1196/2362] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:17:45.857366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:45.867683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88f223-7d85-44e1-b555-d4954cf30f39 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:45.872342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:45.872973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.873255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.873532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] PATH_INFO: `/v3/projects/bc60004f9ad441568f4b3ba4bc6cb667/users/f0a60f5c6e9e4116bd4c50eb0f8ed962/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:45.887836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc60004f9ad441568f4b3ba4bc6cb667, user_id=f0a60f5c6e9e4116bd4c50eb0f8ed962, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:45.890543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:45.974394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1167/2363] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:17:45 2022] PUT /identity/v3/projects/bc60004f9ad441568f4b3ba4bc6cb667/users/f0a60f5c6e9e4116bd4c50eb0f8ed962/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:45.985257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:45.985536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:45.985811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.014151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] MFA Rules not processed for user `f0a60f5c6e9e4116bd4c50eb0f8ed962`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:46.015379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:46.080539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1197/2364] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:46.085530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.105953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:46.145370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0f73f-d117-46b7-bad6-a937360b4acc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.150737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.151353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.151632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.151905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.152723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.154777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.233475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1168/2365] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:17:46 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:17:46.238013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.249314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b9fd-29c9-4bc5-9aee-8134804167f5 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.263392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:46.309156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.309779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.310062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.310337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.310958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.312951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.390768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1198/2366] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:17:46 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:17:46.394592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.405647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d4604-a8ea-446e-843b-3c48f662c054 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.410356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.410952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.411265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.411539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.412207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.413980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.424554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1169/2367] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:17:46.428860 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.438692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac8356-2a28-48e0-ac1a-5524f1f0d1f5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.443268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.444694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.444965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.445233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] PATH_INFO: `/v3/projects/59bdc43d265a4274bd210c64d34fbd1f/users/d6e11a90ebd2488093d6d3ffe946fd90/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.460130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bdc43d265a4274bd210c64d34fbd1f, user_id=d6e11a90ebd2488093d6d3ffe946fd90, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.462767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.534511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1199/2368] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:17:46 2022] PUT /identity/v3/projects/59bdc43d265a4274bd210c64d34fbd1f/users/d6e11a90ebd2488093d6d3ffe946fd90/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:46.538536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.548622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adde187-ce00-4c47-a66e-79c9dfed8a8c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.606661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.607301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.607923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.607923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.608478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.610096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bb68d-acce-4ce4-ab5d-3720e320aec2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.621679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1170/2369] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:17:46.626048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.637420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7a5e-30f1-4b06-b4af-b24ce28262f7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.642155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.642742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.643023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.643339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] PATH_INFO: `/v3/projects/59bdc43d265a4274bd210c64d34fbd1f/users/d6e11a90ebd2488093d6d3ffe946fd90/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.645232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bdc43d265a4274bd210c64d34fbd1f, user_id=d6e11a90ebd2488093d6d3ffe946fd90, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.647954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.658153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1200/2370] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:17:46 2022] PUT /identity/v3/projects/59bdc43d265a4274bd210c64d34fbd1f/users/d6e11a90ebd2488093d6d3ffe946fd90/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:17:46.672412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.672699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.672973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.704215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] MFA Rules not processed for user `d6e11a90ebd2488093d6d3ffe946fd90`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:17:46.704410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:46.776313 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1171/2371] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:17:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:17:46.788166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:46.789916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:17:46.852415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a211d-70ca-4196-bd66-79203164ad1c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:46.856054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:46.856654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:46.856931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:46.857198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:46.866829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:46.868281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:46.877654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1201/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:47.108550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:17:47.119265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d190d2-74d1-44c8-9cb1-8bb6a28c39d0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:17:47.122736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:17:47.123402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:17:47.123682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:17:47.123950 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:17:47.136523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:47.185764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:17:47.188091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9019ad2-5ad8-468b-b0fb-442b6773d5a0 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:17:47.197833 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1172/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:17:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:17:56.873587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:17:57.276242 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=21 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:00.741907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:00.803661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8264ff-6174-4513-bbea-5774823b677d service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:00.808266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:00.808883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:00.810247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:00.810247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] PATH_INFO: `/v3/users/f0a60f5c6e9e4116bd4c50eb0f8ed962` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:00.810498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a60f5c6e9e4116bd4c50eb0f8ed962)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:00.812461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:01.010028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a60f5c6e9e4116bd4c50eb0f8ed962'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.022234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a60f5c6e9e4116bd4c50eb0f8ed962'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.029203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a60f5c6e9e4116bd4c50eb0f8ed962'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.029881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a60f5c6e9e4116bd4c50eb0f8ed962'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.030481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a60f5c6e9e4116bd4c50eb0f8ed962'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.103159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1202/2374] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:00 2022] DELETE /identity/v3/users/f0a60f5c6e9e4116bd4c50eb0f8ed962 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:18:01.107599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:01.108065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:01.108163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:01.127714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:01.128911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:01.188488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1173/2375] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:01.198246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:01.199670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:01.257730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ad9c-1c31-45b7-bb75-907cd0668875 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:01.261267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:01.261864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:01.262140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:01.262410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:01.271340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:01.273130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:01.282377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1203/2376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:01.681379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:01.682910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:01.696970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f5fa7-af7f-4c7f-b6ed-cc0e2cbd9a09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:01.701933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:01.702571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:01.702851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:01.703952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] PATH_INFO: `/v3/projects/bc60004f9ad441568f4b3ba4bc6cb667` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:01.705312 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc60004f9ad441568f4b3ba4bc6cb667)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:01.707664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:01.822921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc60004f9ad441568f4b3ba4bc6cb667', 'request_id': 'req-ca518956-6baa-4e98-8cd4-b70a875de4f4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.828604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc60004f9ad441568f4b3ba4bc6cb667', 'request_id': 'req-ca518956-6baa-4e98-8cd4-b70a875de4f4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.835351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc60004f9ad441568f4b3ba4bc6cb667', 'request_id': 'req-ca518956-6baa-4e98-8cd4-b70a875de4f4'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.936817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] The token cache is being invalidate because project bc60004f9ad441568f4b3ba4bc6cb667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:01.937102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.937718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:01.939350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1174/2377] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:01 2022] DELETE /identity/v3/projects/bc60004f9ad441568f4b3ba4bc6cb667 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:01.944475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:01.954052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aeeed-980d-4911-9f3d-e3f8afa86e49 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:01.972285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:02.010311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:02.012376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:02.012376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:02.012376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] PATH_INFO: `/v3/users/d6e11a90ebd2488093d6d3ffe946fd90` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:02.012673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e11a90ebd2488093d6d3ffe946fd90)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:02.014490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:02.197579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e11a90ebd2488093d6d3ffe946fd90'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.210136 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e11a90ebd2488093d6d3ffe946fd90'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.217350 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e11a90ebd2488093d6d3ffe946fd90'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.218030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e11a90ebd2488093d6d3ffe946fd90'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.218718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0bc6505d-e612-4359-a10f-453d73a0e83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e11a90ebd2488093d6d3ffe946fd90'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.280027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1204/2378] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:01 2022] DELETE /identity/v3/users/d6e11a90ebd2488093d6d3ffe946fd90 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:02.694343 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:02.695811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:02.711017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518956-6baa-4e98-8cd4-b70a875de4f4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:02.768223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:02.768841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:02.769119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:02.769389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] PATH_INFO: `/v3/projects/59bdc43d265a4274bd210c64d34fbd1f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:02.770471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bdc43d265a4274bd210c64d34fbd1f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:02.773059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:02.876053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:02.892144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bdc43d265a4274bd210c64d34fbd1f', 'request_id': 'req-f0ea4709-5321-4c18-8bfa-6387ab7f2913'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.898478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bdc43d265a4274bd210c64d34fbd1f', 'request_id': 'req-f0ea4709-5321-4c18-8bfa-6387ab7f2913'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.904541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bdc43d265a4274bd210c64d34fbd1f', 'request_id': 'req-f0ea4709-5321-4c18-8bfa-6387ab7f2913'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.982719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] The token cache is being invalidate because project 59bdc43d265a4274bd210c64d34fbd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:02.983119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.983810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0ea4709-5321-4c18-8bfa-6387ab7f2913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:02.985308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1175/2379] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:02 2022] DELETE /identity/v3/projects/59bdc43d265a4274bd210c64d34fbd1f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:18:02.991123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:02.991632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:02.991765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.013169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:03.014370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:03.075212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1205/2380] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:03.083229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.083510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.083801 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.103071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:03.104288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:03.112366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1176/2381] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:03.117220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.127131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa205f9-f854-47c4-812d-86bc33ab142a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.132369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.132953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.133228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.133495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.134223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:18:03.134523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.136146 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.148272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1206/2382] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:18:03.152494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.161648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4211dea-e965-4a2c-bc54-b98847f163c2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.167059 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.167709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.167994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.168268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.168900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.170846 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.231387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1177/2383] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/projects => generated 401 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:03.235251 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.245797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb8fe-6247-4942-be68-637b59e801dd admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.273109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:03.302691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.303454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.303782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.304067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.304894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.307277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.371697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1207/2384] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/users => generated 301 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:18:03.375905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.386650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d37dd-1383-4821-b3ed-09b7ce03c7b9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.391295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.391885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.392167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.392441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.393077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.394685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.405198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1178/2385] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:03.409105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.419189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1016e-6d67-4b21-9348-1fd832548bc6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.423827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.424411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.424689 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.424960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.425612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.427240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e67efa-a44e-48b4-bee5-b148f653c38c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.437979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1208/2386] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:03.442017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.452435 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2b9c8-f859-4b10-8649-c6c53cb26c9a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.457018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.457647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.457926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.458198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] PATH_INFO: `/v3/projects/2bc560f95385484ab8d311937f6fe932/users/a90ec8d2ea57461abed7cdb4a106023a/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.473141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc560f95385484ab8d311937f6fe932, user_id=a90ec8d2ea57461abed7cdb4a106023a, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.476091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.560475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1179/2387] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:03 2022] PUT /identity/v3/projects/2bc560f95385484ab8d311937f6fe932/users/a90ec8d2ea57461abed7cdb4a106023a/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:03.572805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.573084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.573379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.601557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] MFA Rules not processed for user `a90ec8d2ea57461abed7cdb4a106023a`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:03.602758 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:03.666543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1209/2388] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:03.671030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.731725 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02769c-6d2a-401e-917e-fa227d5c4aa3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.736370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.736985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.737247 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.737520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.738150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.740907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.828971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1180/2389] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/projects => generated 401 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:03.832949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.844504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5bec3-fee8-49c0-834e-b91fed82ba3b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.901062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.901664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.901947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.902216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.902842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:03.904806 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:03.976392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1210/2390] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:03 2022] POST /identity/v3/users => generated 307 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:18:03.980396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:03.991572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94246e6-02f7-49e6-a3dd-b3f1bd7842a8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:03.996515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:03.997115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:03.997394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:03.997771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:03.998416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.000052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.012761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1181/2391] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:04.016934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:04.027434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc510e-45c8-4032-89d0-d0d7c09d8064 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:04.032025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:04.032637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.032912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.033179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] PATH_INFO: `/v3/projects/b25bcff3bdc54f5ab63976e5add78bed/users/994b08a1ba7045dcaa91114532676157/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.048287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25bcff3bdc54f5ab63976e5add78bed, user_id=994b08a1ba7045dcaa91114532676157, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.050966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.125766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1211/2392] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:04 2022] PUT /identity/v3/projects/b25bcff3bdc54f5ab63976e5add78bed/users/994b08a1ba7045dcaa91114532676157/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:04.129616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:04.139571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092b6c-9ee4-4410-a7eb-8df2f4366b28 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:04.158718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:04.194327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:04.194912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.195215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.195483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.196102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.197769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c422d59-1954-492f-a398-06d594007709 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.207544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1182/2393] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:04.211572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:04.216970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:04.222231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b59e5e-b379-4bea-87f6-20ab63770e7b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:04.227651 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:04.228290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.228566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.228837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] PATH_INFO: `/v3/projects/b25bcff3bdc54f5ab63976e5add78bed/users/994b08a1ba7045dcaa91114532676157/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.235889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25bcff3bdc54f5ab63976e5add78bed, user_id=994b08a1ba7045dcaa91114532676157, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.238828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.250302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1212/2394] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:04 2022] PUT /identity/v3/projects/b25bcff3bdc54f5ab63976e5add78bed/users/994b08a1ba7045dcaa91114532676157/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:04.261007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.261230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.261502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.289150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] MFA Rules not processed for user `994b08a1ba7045dcaa91114532676157`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:04.290341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:04.357704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1183/2395] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:04 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:04.370256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:04.372058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:04.429728 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2f26-d2c9-4ade-8999-4a7115b7e4b1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:04.433300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:04.433891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.435664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.435664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.443572 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.445041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.454418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1213/2396] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:04 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:04.610431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:04.620358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1614f-2616-47ba-b4e8-16d09d7cfb3e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:04.624027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:04.624615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:04.624895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:04.625169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:04.680865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:04.682230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748405d-ea8e-4692-bf81-64aa3a199487 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:04.690968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1184/2397] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:04 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:09.620428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:09.683236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a077-efc4-49e4-83ba-dd0ada9d9df7 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:09.687747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:09.688360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:09.688642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:09.688912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] PATH_INFO: `/v3/users/a90ec8d2ea57461abed7cdb4a106023a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:09.690019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90ec8d2ea57461abed7cdb4a106023a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:09.691993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:09.829586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90ec8d2ea57461abed7cdb4a106023a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:09.842132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90ec8d2ea57461abed7cdb4a106023a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:09.850033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90ec8d2ea57461abed7cdb4a106023a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:09.850661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90ec8d2ea57461abed7cdb4a106023a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:09.851323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90ec8d2ea57461abed7cdb4a106023a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:09.913951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1214/2398] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:09 2022] DELETE /identity/v3/users/a90ec8d2ea57461abed7cdb4a106023a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:09.918613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:09.919136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:09.919642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:09.938532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:09.939844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:09.981101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:10.002317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1185/2399] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:10.013019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:10.014481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:10.029175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:10.076468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f1a7e-4917-4daa-872c-e6419c55008a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:10.080257 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:10.080871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:10.081149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:10.081425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:10.091359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:10.093678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:10.103028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1215/2400] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:10.533011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:10.534447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:10.547368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836e21c-68c2-4a19-b7dc-d737d7777d98 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:10.557960 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:10.563186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:10.563539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:10.563787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] PATH_INFO: `/v3/projects/2bc560f95385484ab8d311937f6fe932` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:10.565107 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc560f95385484ab8d311937f6fe932)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:10.567091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:10.726910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc560f95385484ab8d311937f6fe932', 'request_id': 'req-b87f737c-ce4e-41b2-ac7c-54996c062411'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:10.732555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc560f95385484ab8d311937f6fe932', 'request_id': 'req-b87f737c-ce4e-41b2-ac7c-54996c062411'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:10.739258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc560f95385484ab8d311937f6fe932', 'request_id': 'req-b87f737c-ce4e-41b2-ac7c-54996c062411'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:10.832949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] The token cache is being invalidate because project 2bc560f95385484ab8d311937f6fe932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:10.833791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:10.834561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:10.836100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1186/2401] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:10 2022] DELETE /identity/v3/projects/2bc560f95385484ab8d311937f6fe932 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:10.840186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:10.849876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7d0b5-8e6a-423e-bc60-642afa46b954 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:10.908956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:10.909660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:10.910036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:10.910740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] PATH_INFO: `/v3/users/994b08a1ba7045dcaa91114532676157` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:10.911819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994b08a1ba7045dcaa91114532676157)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:10.913851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:11.144755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994b08a1ba7045dcaa91114532676157'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.157290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994b08a1ba7045dcaa91114532676157'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.164736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994b08a1ba7045dcaa91114532676157'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.165455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994b08a1ba7045dcaa91114532676157'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.166068 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ff1042d1-f5f3-49a5-9d23-576fcb8f4236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994b08a1ba7045dcaa91114532676157'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.240075 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1216/2402] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:10 2022] DELETE /identity/v3/users/994b08a1ba7045dcaa91114532676157 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:11.629682 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:11.631598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:11.645292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f737c-ce4e-41b2-ac7c-54996c062411 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:11.704904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:11.705533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:11.705843 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:11.706140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] PATH_INFO: `/v3/projects/b25bcff3bdc54f5ab63976e5add78bed` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:11.707300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25bcff3bdc54f5ab63976e5add78bed)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:11.709567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:11.815236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25bcff3bdc54f5ab63976e5add78bed', 'request_id': 'req-e87ddf53-1646-4585-931b-45a710117e37'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.820896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25bcff3bdc54f5ab63976e5add78bed', 'request_id': 'req-e87ddf53-1646-4585-931b-45a710117e37'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.827058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25bcff3bdc54f5ab63976e5add78bed', 'request_id': 'req-e87ddf53-1646-4585-931b-45a710117e37'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.907666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:11.912876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] The token cache is being invalidate because project b25bcff3bdc54f5ab63976e5add78bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:11.913162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.916342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e87ddf53-1646-4585-931b-45a710117e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:11.916342 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1187/2403] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:11 2022] DELETE /identity/v3/projects/b25bcff3bdc54f5ab63976e5add78bed => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:11.920379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:11.920859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:11.920966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:11.940785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:11.949580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:11.971754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:12.002822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1217/2404] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:12.010963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.011362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.011800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.032630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:12.033813 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:12.042879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1188/2405] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:12.047163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.056782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f55e8-6d7a-48ae-b731-00be97b257e4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.061252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.061831 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.062100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.062364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.062991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:18:12.063317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.064888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.076174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1218/2406] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:18:12.080296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.090094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb767780-740b-49f1-b83a-bddc868186e2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.094757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.095375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.095655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.095933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.096572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.098450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.188936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1189/2407] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:12.193769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.203835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b33c9f-b91b-40d4-ab1f-8a0b67d6ed07 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.260362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.260885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.261158 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.261426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.262030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.263958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.388009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1219/2408] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:12.392218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.403758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ef05d-8bba-4bb3-888b-5119742470f6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.408259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.408874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.409195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.409463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.410093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.411808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.421619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1190/2409] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:12.425590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.435401 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5747b-e3ab-46cf-90e5-b816e32120a3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.439980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.440563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.440838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.441105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.441733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.443933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e31a4-5d99-476a-9549-854c64caf2d8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.454154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1220/2410] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:12.459044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.468830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247e541-ae58-4409-8c58-5a999c21a0eb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.473573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.474197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.474473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.474743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] PATH_INFO: `/v3/projects/aea3bcc83b76477d87f67207dd36f356/users/cb356ab0f3e84f51a0ab2be397dceb26/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.490250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea3bcc83b76477d87f67207dd36f356, user_id=cb356ab0f3e84f51a0ab2be397dceb26, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.492963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.587763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1191/2411] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:12 2022] PUT /identity/v3/projects/aea3bcc83b76477d87f67207dd36f356/users/cb356ab0f3e84f51a0ab2be397dceb26/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:12.599096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.599464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.599737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.628053 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] MFA Rules not processed for user `cb356ab0f3e84f51a0ab2be397dceb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:12.629268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:12.693017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1221/2412] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:12.697475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.758642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8aa16-64fb-49f9-9051-d0b73efa8087 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.763304 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.763896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.764172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.764441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.765065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.766962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:12.851281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1192/2413] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:12.855322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:12.866698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964764bd-b427-473f-9fe1-7d214cf51c0d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:12.892142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:12.920451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:12.921091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:12.921309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:12.921576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:12.922204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:12.924820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.050638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1222/2414] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:12 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:13.055456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.066040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4b32a-cfe9-416c-a413-62dbc3dd4f16 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.072576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.076876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.076876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.076876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.076876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.076876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.086988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1193/2415] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:13.091259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.102086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6c0e8-26c4-40fe-8d42-e37e7fa6f399 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.106647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.111093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.111093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.111093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] PATH_INFO: `/v3/projects/19886687efc7447796b9524076aaf5e8/users/31c89a5e4c0447bca94fb7eb381fbf0e/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.123314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19886687efc7447796b9524076aaf5e8, user_id=31c89a5e4c0447bca94fb7eb381fbf0e, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.125957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.233754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1223/2416] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:13 2022] PUT /identity/v3/projects/19886687efc7447796b9524076aaf5e8/users/31c89a5e4c0447bca94fb7eb381fbf0e/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:18:13.238108 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.290122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02943f89-4a77-421d-b305-aaeafeb10f83 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.315360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:13.345437 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.346036 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.350729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.350729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.350729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.350729 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df13f79-facb-46e8-b1af-a0a686fad02e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.360176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1194/2417] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:13.365213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.408166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1182162-aa9f-48e5-a188-b188c5bacb9c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.413424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.414059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.414344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.414613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] PATH_INFO: `/v3/projects/19886687efc7447796b9524076aaf5e8/users/31c89a5e4c0447bca94fb7eb381fbf0e/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.416546 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19886687efc7447796b9524076aaf5e8, user_id=31c89a5e4c0447bca94fb7eb381fbf0e, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.419220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.429273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1224/2418] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:13 2022] PUT /identity/v3/projects/19886687efc7447796b9524076aaf5e8/users/31c89a5e4c0447bca94fb7eb381fbf0e/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:13.439574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.439862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.440142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.477420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] MFA Rules not processed for user `31c89a5e4c0447bca94fb7eb381fbf0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:13.478610 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:13.544558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1195/2419] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:13.561249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.562991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:13.635988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa6981-4ad5-4d33-98ae-888431f1d8e3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.639695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.640326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.640609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.640898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.650407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.651881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.661824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1225/2420] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:13.762287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:13.772433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71011a-483e-4c05-b5b2-ff879d3739bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:13.775976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:13.776567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:13.776850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:13.777123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:13.832581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:13.834629 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97deb0d8-6306-422d-b137-3762b39d154e service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:13.843371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1196/2421] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:17.955072 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:17.983279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:18.020180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9c932-4f4e-43c3-b9c5-ff730581b6b2 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:18.025362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:18.025976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:18.026254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:18.026528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] PATH_INFO: `/v3/users/cb356ab0f3e84f51a0ab2be397dceb26` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:18.027657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb356ab0f3e84f51a0ab2be397dceb26)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:18.029578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:18.200378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb356ab0f3e84f51a0ab2be397dceb26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:18.212499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb356ab0f3e84f51a0ab2be397dceb26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:18.219618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb356ab0f3e84f51a0ab2be397dceb26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:18.220289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb356ab0f3e84f51a0ab2be397dceb26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:18.220912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb356ab0f3e84f51a0ab2be397dceb26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:18.307405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1226/2422] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:17 2022] DELETE /identity/v3/users/cb356ab0f3e84f51a0ab2be397dceb26 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:18.312040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:18.312577 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:18.312785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:18.341688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:18.342911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:18.373565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:18.402244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1197/2423] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:18.412744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:18.414695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:18.472768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88fb20-e229-4ace-893b-e8dcf299c851 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:18.476902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:18.477491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:18.477765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:18.478029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:18.487161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:18.488656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:18.498668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1227/2424] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:18.953993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:18.955452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:18.968893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109f6e-5748-461e-8941-679c80efe521 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:18.973837 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:18.974417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:18.974723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:18.974955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] PATH_INFO: `/v3/projects/aea3bcc83b76477d87f67207dd36f356` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:18.976110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea3bcc83b76477d87f67207dd36f356)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:18.978195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:19.118919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea3bcc83b76477d87f67207dd36f356', 'request_id': 'req-fe9be954-03b5-46c4-b92a-7c3e67902bf9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.124558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea3bcc83b76477d87f67207dd36f356', 'request_id': 'req-fe9be954-03b5-46c4-b92a-7c3e67902bf9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.131490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea3bcc83b76477d87f67207dd36f356', 'request_id': 'req-fe9be954-03b5-46c4-b92a-7c3e67902bf9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.219787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] The token cache is being invalidate because project aea3bcc83b76477d87f67207dd36f356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:19.220072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.220701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.222185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1198/2425] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:18 2022] DELETE /identity/v3/projects/aea3bcc83b76477d87f67207dd36f356 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:19.228414 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:19.236796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc4c9c-9a84-4580-b106-55edccc80c26 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:19.280789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:19.293574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:19.294196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:19.294473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:19.294744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] PATH_INFO: `/v3/users/31c89a5e4c0447bca94fb7eb381fbf0e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:19.295883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c89a5e4c0447bca94fb7eb381fbf0e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:19.297802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:19.542204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c89a5e4c0447bca94fb7eb381fbf0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.557266 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c89a5e4c0447bca94fb7eb381fbf0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.578833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c89a5e4c0447bca94fb7eb381fbf0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.578833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c89a5e4c0447bca94fb7eb381fbf0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.578833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-55e47a1b-6b85-4a86-adc6-6e7722559818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c89a5e4c0447bca94fb7eb381fbf0e'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:19.649656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1228/2426] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:19 2022] DELETE /identity/v3/users/31c89a5e4c0447bca94fb7eb381fbf0e => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:20.085371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:20.087432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:20.114901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be954-03b5-46c4-b92a-7c3e67902bf9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:20.173258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:20.173883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.174172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.174448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] PATH_INFO: `/v3/projects/19886687efc7447796b9524076aaf5e8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.179840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19886687efc7447796b9524076aaf5e8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:20.181984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:20.328988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19886687efc7447796b9524076aaf5e8', 'request_id': 'req-b310b7e8-1abc-4ca4-9b74-32a7aca28578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:20.335223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19886687efc7447796b9524076aaf5e8', 'request_id': 'req-b310b7e8-1abc-4ca4-9b74-32a7aca28578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:20.336314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:20.341855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19886687efc7447796b9524076aaf5e8', 'request_id': 'req-b310b7e8-1abc-4ca4-9b74-32a7aca28578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:20.467495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] The token cache is being invalidate because project 19886687efc7447796b9524076aaf5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:20.467878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:20.468699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b310b7e8-1abc-4ca4-9b74-32a7aca28578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:20.470371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1199/2427] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:20 2022] DELETE /identity/v3/projects/19886687efc7447796b9524076aaf5e8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:20.476614 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.477209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.477368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.535963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:20.538129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:20.602945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1229/2428] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:20.611907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.612318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.612737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.634656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:20.635898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:20.643979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1200/2429] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:20.650223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:20.658446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc18b-6cd6-4676-858c-b58a1b440cd8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:20.663929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:20.664520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.664801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.665067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.665696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:18:20.665994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:20.667615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:20.679748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1230/2430] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:18:20.684058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:20.693438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6076a5c-d8d0-4997-ad51-5baa6b5038c0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:20.698102 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:20.698689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.698967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.699276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.699907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:20.701881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:20.818469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1201/2431] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:20 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:20.819770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:20.829902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b47208-cd41-4eda-8f2e-e49e2fefcc90 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:20.856942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:20.886805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:20.887460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:20.887753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:20.888223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:20.888902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:20.891449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.012348 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1231/2432] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:20 2022] POST /identity/v3/users => generated 305 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:21.013579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.024836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075bac4-3807-40d3-8c3e-cab23607d925 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.030055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.030782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.031167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.031533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.032204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.033797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.049952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1202/2433] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:21.054154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.064483 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b3e14-14e3-4115-a42b-19c080db03cc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.069412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.070562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.070562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.070562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.071325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.073774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1bb62-1d3a-4287-8989-f204bae8eb4d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.087922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1232/2434] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:21.089411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.100943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aea69-8a1e-4ba9-bb84-fc4603e3e0e3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.105800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.111207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.111207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.111207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] PATH_INFO: `/v3/projects/89c2da653dbf4df7b9f10d5ab85bc48d/users/4adb31b0c09543068f5c3487a42bc6a7/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.122376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c2da653dbf4df7b9f10d5ab85bc48d, user_id=4adb31b0c09543068f5c3487a42bc6a7, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.125068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.211620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1203/2435] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:21 2022] PUT /identity/v3/projects/89c2da653dbf4df7b9f10d5ab85bc48d/users/4adb31b0c09543068f5c3487a42bc6a7/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:21.222284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.222649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.222930 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.250754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] MFA Rules not processed for user `4adb31b0c09543068f5c3487a42bc6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:21.252021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:21.315872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1233/2436] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:21 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:21.320971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.382089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d052-acc5-4e6e-acad-9578ae659d0c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.386625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.387241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.387539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.387796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.388417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.390324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.507616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1204/2437] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:21 2022] POST /identity/v3/projects => generated 407 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:21.511617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.522754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684a11d-b6f5-446a-a266-084e0fa5cd3c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.579006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.579655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.579934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.580209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.580828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.582769 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.695609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1234/2438] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:21 2022] POST /identity/v3/users => generated 310 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:21.699285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.709906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a365-a72e-438b-9ff6-e2695f5b1bcf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.714520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.715137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.715414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.715690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.716324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.717929 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.728450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1205/2439] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:21.732521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.743307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1543b-dcbb-4af6-bf67-1bf9e75225fc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.747811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.748463 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.748737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.749006 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] PATH_INFO: `/v3/projects/711011136e8742deb5c2ad4729fa879e/users/23b04db9c7844a5b858daa3afcc798e2/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.765090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] RBAC: Authorizing `identity:create_grant(project_id=711011136e8742deb5c2ad4729fa879e, user_id=23b04db9c7844a5b858daa3afcc798e2, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.767777 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.820485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1235/2440] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:21 2022] PUT /identity/v3/projects/711011136e8742deb5c2ad4729fa879e/users/23b04db9c7844a5b858daa3afcc798e2/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:21.824585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.834655 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ed4b-9793-4482-975c-9b427f01504e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.884006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:21.889580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.890880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.890880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.890880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.891401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.892985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1aceab-e874-4382-bb44-04618539941e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.902508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1206/2441] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:21.906770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:21.912237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:21.917388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0b6c-b913-4717-bcea-84443ac1537c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:21.921971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:21.922628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.922901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.923209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] PATH_INFO: `/v3/projects/711011136e8742deb5c2ad4729fa879e/users/23b04db9c7844a5b858daa3afcc798e2/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.925064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711011136e8742deb5c2ad4729fa879e, user_id=23b04db9c7844a5b858daa3afcc798e2, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:21.928200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:21.938186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1236/2442] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:21 2022] PUT /identity/v3/projects/711011136e8742deb5c2ad4729fa879e/users/23b04db9c7844a5b858daa3afcc798e2/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:21.948876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:21.949097 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:21.949371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:21.976406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] MFA Rules not processed for user `23b04db9c7844a5b858daa3afcc798e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:21.977592 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:22.045885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1207/2443] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:21 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:22.057093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:22.058820 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:22.119742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5fbb-c698-4bc3-b2df-26cb593416b0 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:22.123512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:22.124139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:22.124412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:22.124680 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:22.134127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:22.135773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:22.145857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1237/2444] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:22 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:22.533911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:22.544081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27fd9-1276-466d-b5c1-fa112a7a47e1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:22.547920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:22.548674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:22.548959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:22.549235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:22.613364 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:22.615203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a8623-5a0a-475e-9cb5-48e168e58733 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:22.624546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1208/2445] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:22 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:29.911515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:29.911515 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:30.002910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5331646-79e2-4e38-9a0d-57a1ab70234f service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:30.008067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:30.008671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:30.008945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:30.009208 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:30.010976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:30.023552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:30.027385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:30.036081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1238/2446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:32.620211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=65 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:33.830627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:33.831243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:33.831338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:33.851831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] MFA Rules not processed for user `02240dc931e140498d82c6d91efb60c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:33.853106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:33.927257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1209/2447] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:34.262947 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:34.264468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:34.342812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:34.353289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f44c8-8b3c-4476-8985-fc1666f61175 service placement] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:34.358123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:34.359645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:34.360221 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:34.360698 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:34.362961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:34.375746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:34.377201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f3f4d-7210-4ed7-92e1-5cf89c80c693 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:34.388398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1239/2448] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:34.912003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:35.002016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d18e9-b8f0-41d4-899f-df9682f95487 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:35.007583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:35.008213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:35.008486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:35.008926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] PATH_INFO: `/v3/users/4adb31b0c09543068f5c3487a42bc6a7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:35.010044 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adb31b0c09543068f5c3487a42bc6a7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:35.012068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:35.262911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adb31b0c09543068f5c3487a42bc6a7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:35.285013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adb31b0c09543068f5c3487a42bc6a7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:35.383420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adb31b0c09543068f5c3487a42bc6a7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:35.384096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adb31b0c09543068f5c3487a42bc6a7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:35.384708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adb31b0c09543068f5c3487a42bc6a7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:35.404365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1210/2449] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:34 2022] DELETE /identity/v3/users/4adb31b0c09543068f5c3487a42bc6a7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:35.409877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:35.410349 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:35.410569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:35.429506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:35.430705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:35.491876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1240/2450] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:35.501910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:35.503894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:35.531859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:35.562099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97531-94b1-4e44-adcd-3b295587cecd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:35.566680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:35.567331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:35.567631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:35.567883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:35.577468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:35.579392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:35.589548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1211/2451] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:18:36.042750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:36.044235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:36.057869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815586d-ee41-43ce-9f14-2b982e61eecf None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:36.062875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:36.063554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:36.068915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:36.068915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] PATH_INFO: `/v3/projects/89c2da653dbf4df7b9f10d5ab85bc48d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:36.068915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c2da653dbf4df7b9f10d5ab85bc48d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:36.068915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:36.184590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c2da653dbf4df7b9f10d5ab85bc48d', 'request_id': 'req-194a8b51-899c-4106-a09f-26ead4bdb846'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.252739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c2da653dbf4df7b9f10d5ab85bc48d', 'request_id': 'req-194a8b51-899c-4106-a09f-26ead4bdb846'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.259553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c2da653dbf4df7b9f10d5ab85bc48d', 'request_id': 'req-194a8b51-899c-4106-a09f-26ead4bdb846'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.269969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] The token cache is being invalidate because project 89c2da653dbf4df7b9f10d5ab85bc48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:36.270259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.270931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.272454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1241/2452] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:36 2022] DELETE /identity/v3/projects/89c2da653dbf4df7b9f10d5ab85bc48d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:36.276505 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:36.286710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a4ba-7957-42c2-84f2-a9f14d8ae423 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:36.349784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:36.350567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:36.350854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:36.351149 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] PATH_INFO: `/v3/users/23b04db9c7844a5b858daa3afcc798e2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:36.352440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b04db9c7844a5b858daa3afcc798e2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:36.354821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:36.632703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b04db9c7844a5b858daa3afcc798e2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.645438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b04db9c7844a5b858daa3afcc798e2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.715817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b04db9c7844a5b858daa3afcc798e2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.716439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b04db9c7844a5b858daa3afcc798e2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.717098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa504ec0-03b3-4523-8ebb-c50c5043f209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b04db9c7844a5b858daa3afcc798e2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:36.732218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1212/2453] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:36 2022] DELETE /identity/v3/users/23b04db9c7844a5b858daa3afcc798e2 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:18:37.197371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:37.198815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:37.213099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8b51-899c-4106-a09f-26ead4bdb846 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:37.280827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:37.281461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:37.281740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:37.282011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] PATH_INFO: `/v3/projects/711011136e8742deb5c2ad4729fa879e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:37.283144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711011136e8742deb5c2ad4729fa879e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:37.286013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:37.297513 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:37.440468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711011136e8742deb5c2ad4729fa879e', 'request_id': 'req-2e579fef-45f3-4346-84d1-39add86f1ef4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:37.532811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711011136e8742deb5c2ad4729fa879e', 'request_id': 'req-2e579fef-45f3-4346-84d1-39add86f1ef4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:37.541091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711011136e8742deb5c2ad4729fa879e', 'request_id': 'req-2e579fef-45f3-4346-84d1-39add86f1ef4'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:37.559565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] The token cache is being invalidate because project 711011136e8742deb5c2ad4729fa879e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:37.559773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:37.560544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-2e579fef-45f3-4346-84d1-39add86f1ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:37.565492 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1242/2454] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:37 2022] DELETE /identity/v3/projects/711011136e8742deb5c2ad4729fa879e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:37.567048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:37.567549 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:37.567663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:37.589367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:37.590603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:37.638477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:37.657241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1213/2455] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:37.724895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:37.725358 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:37.725591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:37.749608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:37.750850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:37.763628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1243/2456] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:37.773221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:37.785413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29362c-1f92-48e0-83dc-c3cabc5b19ad None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:37.793022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:37.793659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:37.794153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:37.794635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:37.795539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:18:37.796073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:37.797897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:37.813836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1214/2457] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:18:37.819178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:37.829142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48e7f9-c521-4986-806d-1b2d8944f44d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:37.836430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:37.836430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:37.836430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:37.836430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:37.836801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:37.838436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:37.935441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1244/2458] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:37 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:37.940533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:37.950632 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4903047-2d3a-4ae6-ae78-069c87da69c0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:38.007949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:38.008553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.008831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.009103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.009726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:38.011905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:38.119408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1215/2459] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:37 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:38.120972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:38.133217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f3d14-eeaa-4294-8969-ed9acf463f3a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:38.139275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:38.140086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.140472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.140849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.141891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:38.146621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:38.161782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1245/2460] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:38.165660 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:38.180310 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32691a-b7e9-4001-a9f0-59daa1a546df admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:38.185588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:38.186376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.186709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.187178 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.187836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:38.189605 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38697fce-29c6-4c27-ae1b-068f0641c6f8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:38.202842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1216/2461] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:38.207276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:38.219028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e757ae-e5fe-4a65-b567-543fbdc3fdd1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:38.224497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:38.225292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.225669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.226043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] PATH_INFO: `/v3/projects/21ee580ebd554ad9b4e98454d7f373bf/users/e62700cf9b3f4330b3b7b553acdf9107/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.242012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ee580ebd554ad9b4e98454d7f373bf, user_id=e62700cf9b3f4330b3b7b553acdf9107, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:38.244714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:38.333013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1246/2462] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:38 2022] PUT /identity/v3/projects/21ee580ebd554ad9b4e98454d7f373bf/users/e62700cf9b3f4330b3b7b553acdf9107/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:38.338093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297449f-af13-4ada-aa2a-db9f21912e25 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.338449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297449f-af13-4ada-aa2a-db9f21912e25 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.338798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297449f-af13-4ada-aa2a-db9f21912e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.369117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d297449f-af13-4ada-aa2a-db9f21912e25 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:38.369968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d297449f-af13-4ada-aa2a-db9f21912e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:38.436192 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1217/2463] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:38.447468 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:38.449411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:38.512402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce73b49-e0af-44f0-93b5-9496399d19c3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:38.516292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:38.516931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:38.517259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:38.517537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:38.527668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:38.529113 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:38.540040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1247/2464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:39.909366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:39.910320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:39.910829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:39.944070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] MFA Rules not processed for user `e62700cf9b3f4330b3b7b553acdf9107`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:39.945027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:39.998900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:40.031275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1218/2465] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:40.035097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.046478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9604491-952a-44cc-b411-e318943abe90 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.052838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.053682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.057387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.057387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.057387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.057387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.153708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1248/2466] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:18:40 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:40.154996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.166832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bff8a-5d9c-46f0-b006-ede0df3c24ad None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.231749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.232362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.232901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.233226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.233868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.235844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.361638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1219/2467] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:18:40 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:18:40.365774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.376494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776638a-d68e-4f3f-8499-d0327c808feb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.381912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.382539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.382823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.383084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.383740 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.385319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.396654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1249/2468] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:40.402494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.413519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c2689-7146-41cd-b285-8109d05a0216 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.418201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.418835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.419124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.419419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] PATH_INFO: `/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c/users/ba9f34fcfdab4d84a7035759b5a95c8b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.434330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7310bd227f4bc6977b32f77b8d5c6c, user_id=ba9f34fcfdab4d84a7035759b5a95c8b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.437019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.506532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1220/2469] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:40 2022] PUT /identity/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c/users/ba9f34fcfdab4d84a7035759b5a95c8b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:40.511871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.528030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:40.532390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7eca8-5440-4ca8-a6c0-0ed38b0c79a5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.612891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.613606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.613955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.614304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.615025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.617366 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59baf22f-0ce7-4b13-bda6-2311c971581a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.633903 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1250/2470] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:18:40.637762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:40.649144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a3c89-d4f1-491d-b4b8-8478e1bc6fe6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:40.661449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:40.662180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:40.662588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:40.662699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] PATH_INFO: `/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c/users/ba9f34fcfdab4d84a7035759b5a95c8b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:40.664956 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7310bd227f4bc6977b32f77b8d5c6c, user_id=ba9f34fcfdab4d84a7035759b5a95c8b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:40.668202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:40.679762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1221/2471] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:18:40 2022] PUT /identity/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c/users/ba9f34fcfdab4d84a7035759b5a95c8b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:42.248320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:42.248839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:42.248839 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:42.280646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] MFA Rules not processed for user `ba9f34fcfdab4d84a7035759b5a95c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:42.282427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:42.352121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1251/2472] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:18:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:42.364325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:42.365989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:42.444612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3eca5-4727-4ee2-86b8-837eb6df098e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:42.448218 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:42.448819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:42.449099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:42.449372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:42.479773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:42.508347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:42.509845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20f684-804a-4017-aca3-52a318603d0f service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:42.519277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1222/2473] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:45.295684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:45.305909 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccf6f6-0b66-479a-b652-fad8e796e577 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:45.309509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:45.310093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:45.310365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:45.310631 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:45.320833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:45.324936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:45.332658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1252/2474] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:18:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:18:45.678415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855c357-c925-422c-a927-5867f28aad9a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:45.678866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855c357-c925-422c-a927-5867f28aad9a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:45.678973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855c357-c925-422c-a927-5867f28aad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:45.709077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2855c357-c925-422c-a927-5867f28aad9a None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:18:45.710269 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2855c357-c925-422c-a927-5867f28aad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:45.776240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1223/2475] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:18:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:18:45.940287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:45.941877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:46.012190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4b14-2c56-49b1-93b1-0e529ee88e97 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:46.017240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:46.017836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:46.018112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:46.018384 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:46.026294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:46.027931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:46.036521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1253/2476] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:18:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 09:18:55.246299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855c357-c925-422c-a927-5867f28aad9a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:55.247876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2855c357-c925-422c-a927-5867f28aad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:55.319768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855c357-c925-422c-a927-5867f28aad9a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:55.324306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:55.324933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:55.325217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:55.325494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] PATH_INFO: `/v3/users/e64f94da55234c21889ecf1cd2220ed5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:55.326628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e64f94da55234c21889ecf1cd2220ed5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:55.328696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:55.514705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e64f94da55234c21889ecf1cd2220ed5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:55.526327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:55.528798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e64f94da55234c21889ecf1cd2220ed5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:55.597504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e64f94da55234c21889ecf1cd2220ed5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:55.598068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e64f94da55234c21889ecf1cd2220ed5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:55.598681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e64f94da55234c21889ecf1cd2220ed5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:55.612654 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1224/2477] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:55 2022] DELETE /identity/v3/users/e64f94da55234c21889ecf1cd2220ed5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:55.879078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:55.884728 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:55.899407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a26e90-6567-4201-a653-72cfff8008dd service swift] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:55.967163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886470f-45a6-482e-985b-9510896443db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:55.967797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886470f-45a6-482e-985b-9510896443db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:55.968081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886470f-45a6-482e-985b-9510896443db admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:55.968357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886470f-45a6-482e-985b-9510896443db admin admin] PATH_INFO: `/v3/projects/14201dd9cdf946d2824c31e55c0573f8` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:55.970357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886470f-45a6-482e-985b-9510896443db admin admin] RBAC: Authorizing `identity:delete_project(project_id=14201dd9cdf946d2824c31e55c0573f8)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:55.972446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886470f-45a6-482e-985b-9510896443db admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:56.091297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14201dd9cdf946d2824c31e55c0573f8', 'request_id': 'req-4886470f-45a6-482e-985b-9510896443db'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.180186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14201dd9cdf946d2824c31e55c0573f8', 'request_id': 'req-4886470f-45a6-482e-985b-9510896443db'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.187088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14201dd9cdf946d2824c31e55c0573f8', 'request_id': 'req-4886470f-45a6-482e-985b-9510896443db'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.199135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] The token cache is being invalidate because project 14201dd9cdf946d2824c31e55c0573f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:56.200874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.200874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.201683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1254/2478] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:55 2022] DELETE /identity/v3/projects/14201dd9cdf946d2824c31e55c0573f8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:56.206305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:56.207987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:56.221546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86242fa-2728-494d-879e-6a2b14b44da7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:56.278422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:56.279213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:56.279420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:56.279601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] PATH_INFO: `/v3/users/df0f6118586342a18d14a4f071025933` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:56.280775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0f6118586342a18d14a4f071025933)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:56.283422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:56.516786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0f6118586342a18d14a4f071025933'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.529806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0f6118586342a18d14a4f071025933'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.632714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0f6118586342a18d14a4f071025933'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.636234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0f6118586342a18d14a4f071025933'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.636234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-860e68b6-333b-4517-956f-c4cda3b40097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0f6118586342a18d14a4f071025933'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:56.647589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1225/2479] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:18:56 2022] DELETE /identity/v3/users/df0f6118586342a18d14a4f071025933 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:18:56.909147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:18:56.918744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:18:56.924264 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886470f-45a6-482e-985b-9510896443db admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:18:56.989567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:18:56.990201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:18:56.990478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:18:56.990751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] PATH_INFO: `/v3/projects/1ef6904564574c4dbfc996fb74e05751` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:18:56.991905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef6904564574c4dbfc996fb74e05751)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:18:56.993818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:18:57.075081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:18:57.084176 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef6904564574c4dbfc996fb74e05751', 'request_id': 'req-c6d88042-1a33-4716-9a45-e82d826e5c5c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:57.147205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef6904564574c4dbfc996fb74e05751', 'request_id': 'req-c6d88042-1a33-4716-9a45-e82d826e5c5c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:57.154077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef6904564574c4dbfc996fb74e05751', 'request_id': 'req-c6d88042-1a33-4716-9a45-e82d826e5c5c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:57.165056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] The token cache is being invalidate because project 1ef6904564574c4dbfc996fb74e05751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:18:57.165346 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:57.165979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c6d88042-1a33-4716-9a45-e82d826e5c5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:18:57.167586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1255/2480] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:18:56 2022] DELETE /identity/v3/projects/1ef6904564574c4dbfc996fb74e05751 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:01.884379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:01.884915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:01.884915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:01.886293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1226/2481] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:19:01.897013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:01.897504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:01.897591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:01.938329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] MFA Rules not processed for user `97133f7b0f5b4320a41486752de09440`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:01.939597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:02.002541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1256/2482] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:19:02.015240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:02.017233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:02.034361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbae8b-63d5-492e-82e0-b60430c29c88 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:02.039420 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:02.040022 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:02.040303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:02.040576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:02.050047 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:02.051706 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:02.061719 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1227/2483] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:19:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:19:02.762100 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:02.763601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:02.832637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b2e6c-f401-4455-85ee-bf8f0c92e5b8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:02.837245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:02.837870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:02.838149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:02.838423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] PATH_INFO: `/v3/users/e62700cf9b3f4330b3b7b553acdf9107` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:02.839603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62700cf9b3f4330b3b7b553acdf9107)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:02.841541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:03.004424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62700cf9b3f4330b3b7b553acdf9107'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.005504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:03.017215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62700cf9b3f4330b3b7b553acdf9107'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.023925 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62700cf9b3f4330b3b7b553acdf9107'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.024601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62700cf9b3f4330b3b7b553acdf9107'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.025210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62700cf9b3f4330b3b7b553acdf9107'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.093357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1257/2484] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:19:02 2022] DELETE /identity/v3/users/e62700cf9b3f4330b3b7b553acdf9107 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:03.330809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:03.341634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f37-8807-4ab1-84ca-225446dfa4a5 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:03.387921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:03.402968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:03.403627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:03.403901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:03.404168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] PATH_INFO: `/v3/projects/21ee580ebd554ad9b4e98454d7f373bf` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:03.405257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ee580ebd554ad9b4e98454d7f373bf)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:03.407393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:03.496462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ee580ebd554ad9b4e98454d7f373bf', 'request_id': 'req-5872faea-3c4c-44f2-9f11-438587e3a06b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.502618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ee580ebd554ad9b4e98454d7f373bf', 'request_id': 'req-5872faea-3c4c-44f2-9f11-438587e3a06b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.508786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ee580ebd554ad9b4e98454d7f373bf', 'request_id': 'req-5872faea-3c4c-44f2-9f11-438587e3a06b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.585829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] The token cache is being invalidate because project 21ee580ebd554ad9b4e98454d7f373bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:19:03.586052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.586687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.588248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1228/2485] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:19:03 2022] DELETE /identity/v3/projects/21ee580ebd554ad9b4e98454d7f373bf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:03.592336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:03.594254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:03.607488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78a07-9a24-4ddc-a28c-be654c371056 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:03.663402 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:03.664024 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:03.664302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:03.664575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] PATH_INFO: `/v3/users/ba9f34fcfdab4d84a7035759b5a95c8b` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:03.665695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba9f34fcfdab4d84a7035759b5a95c8b)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:03.667652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:03.858811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba9f34fcfdab4d84a7035759b5a95c8b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.871531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba9f34fcfdab4d84a7035759b5a95c8b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.878867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba9f34fcfdab4d84a7035759b5a95c8b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.879585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba9f34fcfdab4d84a7035759b5a95c8b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.880200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-965b5c0a-ae2a-4a19-a71b-8a889588a59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba9f34fcfdab4d84a7035759b5a95c8b'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:03.957214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1258/2486] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:19:03 2022] DELETE /identity/v3/users/ba9f34fcfdab4d84a7035759b5a95c8b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:19:04.170272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.172240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:04.186191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872faea-3c4c-44f2-9f11-438587e3a06b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.245129 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.245760 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.246043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.246322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] PATH_INFO: `/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.247494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7310bd227f4bc6977b32f77b8d5c6c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.249443 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.379756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7310bd227f4bc6977b32f77b8d5c6c', 'request_id': 'req-b471480a-4d1f-4b85-9cbe-06ba0be1705d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:04.385353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7310bd227f4bc6977b32f77b8d5c6c', 'request_id': 'req-b471480a-4d1f-4b85-9cbe-06ba0be1705d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:04.392406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7310bd227f4bc6977b32f77b8d5c6c', 'request_id': 'req-b471480a-4d1f-4b85-9cbe-06ba0be1705d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:04.486864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] The token cache is being invalidate because project 1f7310bd227f4bc6977b32f77b8d5c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:19:04.487182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:04.487800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b471480a-4d1f-4b85-9cbe-06ba0be1705d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:04.489281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1229/2487] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:19:04 2022] DELETE /identity/v3/projects/1f7310bd227f4bc6977b32f77b8d5c6c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:04.494419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.494966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.494966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.516202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:04.517410 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:04.557374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:04.576815 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1259/2488] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:04.585286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.585598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.585871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.594690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:04.606281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:04.607518 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:04.615466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1230/2489] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:04.619682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.629811 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14dee2a-7be1-4b41-900a-f0016daa06ea None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.634746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.635392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.643518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.643518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.643518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:19:04.643518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.643518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.650575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1260/2490] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:19:04.654769 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.664103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591301eb-53ba-4d79-8359-e8fc51812b64 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.668858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.669446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.669728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.670001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.670622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.673142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.736571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1231/2491] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:19:04 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:19:04.740716 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.751260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1a4d-e6f6-4bb8-b34f-8a6469089f0a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.808353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.808953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.809231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.809504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.810134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.812088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.886805 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1261/2492] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:19:04 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:19:04.888298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.898890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5a5b1-8743-450a-abfd-a546f526c84c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.903650 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.904239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.904558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.904822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.905460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.907150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.917644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1232/2493] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:19:04.921690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.932679 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f488465-d5d3-4ba8-83cb-c7b810670e0e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.937656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.938833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.939134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.939438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.940078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.942120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb947-91c4-4bf0-817c-64916667f457 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:04.952448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1262/2494] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:19:04.956615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:04.966994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4005cb-f5c3-433c-a57d-b3487d0407ff admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:04.972073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:04.972723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:04.973008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:04.973280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] PATH_INFO: `/v3/projects/90cdf897168841589a29b9877469222f/users/3e866802293a472b9dc9396b154eaaac/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:04.989859 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cdf897168841589a29b9877469222f, user_id=3e866802293a472b9dc9396b154eaaac, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:04.992607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:05.066056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1233/2495] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:19:04 2022] PUT /identity/v3/projects/90cdf897168841589a29b9877469222f/users/3e866802293a472b9dc9396b154eaaac/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:19:05.076338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:05.076621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:05.076895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:05.105317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] MFA Rules not processed for user `3e866802293a472b9dc9396b154eaaac`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:05.106524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:05.170119 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1263/2496] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:19:05.181609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:05.183348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:05.246815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa7b8c-9a24-4d13-8615-ad46fa829118 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:05.251302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:05.251995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:05.252278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:05.252551 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:05.261039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:05.262465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf76bd-21f4-4727-a421-fb89a9269503 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:05.271912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1234/2497] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:19:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:19:05.292228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:05.334558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:05.354396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1ef31-bfa8-43e1-80af-31e9807e3cf8 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:05.359129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:05.359757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:05.360739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:05.361018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] PATH_INFO: `/v3/users/3e866802293a472b9dc9396b154eaaac` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:05.362197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e866802293a472b9dc9396b154eaaac)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:05.364730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:05.644792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e866802293a472b9dc9396b154eaaac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:05.656571 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e866802293a472b9dc9396b154eaaac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:05.663382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e866802293a472b9dc9396b154eaaac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:05.664062 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e866802293a472b9dc9396b154eaaac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:05.664683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e866802293a472b9dc9396b154eaaac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:05.734216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1264/2498] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:19:05 2022] DELETE /identity/v3/users/3e866802293a472b9dc9396b154eaaac => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:05.739345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:05.739816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:05.739937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:05.759937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:05.761132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:05.819596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1235/2499] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:05.829357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:05.830738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:05.886877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c840-cd02-45d4-acc0-67a49f7ae468 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:05.890452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:05.891079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:05.891388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:05.891759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:05.900587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:05.901996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50cb22-30e6-468c-be57-3106f66b619f service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:05.911827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1265/2500] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:19:06.380232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:06.381648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:06.395828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635a51b-e6e8-4ba4-880b-837001a2dcab None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:06.400509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:06.401141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:06.405856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:06.405856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] PATH_INFO: `/v3/projects/90cdf897168841589a29b9877469222f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:06.405856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cdf897168841589a29b9877469222f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:06.405856 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:06.521422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:06.528232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cdf897168841589a29b9877469222f', 'request_id': 'req-2abe468a-f072-4635-b3e1-ff046fef0f41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:06.534617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cdf897168841589a29b9877469222f', 'request_id': 'req-2abe468a-f072-4635-b3e1-ff046fef0f41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:06.540869 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cdf897168841589a29b9877469222f', 'request_id': 'req-2abe468a-f072-4635-b3e1-ff046fef0f41'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:06.625765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] The token cache is being invalidate because project 90cdf897168841589a29b9877469222f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:19:06.626046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:06.626675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:19:06.628301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1236/2501] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:19:06 2022] DELETE /identity/v3/projects/90cdf897168841589a29b9877469222f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:19:07.695941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a1c-6181-484e-af2f-d8138271762e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:07.696448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a1c-6181-484e-af2f-d8138271762e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:07.696589 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52765a1c-6181-484e-af2f-d8138271762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:07.715871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-52765a1c-6181-484e-af2f-d8138271762e None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:07.717094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-52765a1c-6181-484e-af2f-d8138271762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:07.772198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1266/2502] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:07.784701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:07.786080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:07.855363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe468a-f072-4635-b3e1-ff046fef0f41 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:07.860264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:07.860854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:07.861130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:07.861399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:07.869417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:07.871008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:07.879221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1237/2503] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:19:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:19:17.764640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:17.876070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=42,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:29.170258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894501-b9f9-4871-ba0f-b6a61c7194b3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:29.170782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894501-b9f9-4871-ba0f-b6a61c7194b3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:29.170886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894501-b9f9-4871-ba0f-b6a61c7194b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:29.190199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e8894501-b9f9-4871-ba0f-b6a61c7194b3 None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:29.196326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e8894501-b9f9-4871-ba0f-b6a61c7194b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:29.199496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1267/2504] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:29.211330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:29.221800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4367-8826-49cf-babb-4d5f9f824043 service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:29.227223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:29.227799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:29.228076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:29.228346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:29.236189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:29.277570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:29.277570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1238/2505] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:19:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:19:39.185364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:39.242999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:48.336713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:48.337229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:48.337229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:48.356714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] MFA Rules not processed for user `e9d0bba06c9c4425a0e96641dab46c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:19:48.357884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:19:48.364961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1268/2506] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:19:48.387027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:19:48.397668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f11fef-b4fa-4df2-8226-6e263ff43c38 service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:19:48.402942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95310969054beab60e99fd71a34aee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:19:48.403567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:19:48.403847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:19:48.404120 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:19:48.412379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:19:48.413980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:19:48.423172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1239/2507] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 09:19:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 09:19:58.350753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:19:58.419906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:10.206013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:10.207590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:10.264957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4015f-d155-44a5-b021-81630fb4b96a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:10.268578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:10.269217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:10.269495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:10.269767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:10.271533 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:10.333343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:10.334757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:10.344813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1269/2508] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:14.001868 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:14.005468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:14.017540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9ebbd-8d26-4dda-8fc3-54eb88ecb90c service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:14.022143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:14.022787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:14.023070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:14.023381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] PATH_INFO: `/v3/users/004ab0c0b138457e9a5e9288d039b932` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:14.039211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004ab0c0b138457e9a5e9288d039b932)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:14.041865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:14.230616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004ab0c0b138457e9a5e9288d039b932'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.242308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004ab0c0b138457e9a5e9288d039b932'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.325136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004ab0c0b138457e9a5e9288d039b932'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.325757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004ab0c0b138457e9a5e9288d039b932'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.326367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004ab0c0b138457e9a5e9288d039b932'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.339747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1240/2509] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:14 2022] DELETE /identity/v3/users/004ab0c0b138457e9a5e9288d039b932 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:14.549373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:14.550898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:14.585587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:14.587057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:14.615460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292cb40-7694-4be1-b364-04e0befd91f6 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:14.658790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e259fb4-94c2-4c6b-984f-754561bd7f54 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:14.663774 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:14.664527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:14.664807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:14.665822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] PATH_INFO: `/v3/projects/f5045a17a253414e9c231bcd0d7d24b8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:14.667404 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:14.668005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:14.668283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:14.668551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:14.670351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:14.678082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5045a17a253414e9c231bcd0d7d24b8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:14.680182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:14.693179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:14.753986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:14.755344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc75dd-618b-4088-94d2-ebeab82ad60a service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:14.776432 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1270/2510] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:14.789467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5045a17a253414e9c231bcd0d7d24b8', 'request_id': 'req-fad90792-ce15-4e85-ac86-7f1eaa10151e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.794534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:14.796791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5045a17a253414e9c231bcd0d7d24b8', 'request_id': 'req-fad90792-ce15-4e85-ac86-7f1eaa10151e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.803712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5045a17a253414e9c231bcd0d7d24b8', 'request_id': 'req-fad90792-ce15-4e85-ac86-7f1eaa10151e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.880411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] The token cache is being invalidate because project f5045a17a253414e9c231bcd0d7d24b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:14.880808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.881469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fad90792-ce15-4e85-ac86-7f1eaa10151e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:14.883159 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1241/2511] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:14 2022] DELETE /identity/v3/projects/f5045a17a253414e9c231bcd0d7d24b8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:20:14.888252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:14.888452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:14.888719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:14.909228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:14.914585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:14.968069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1271/2512] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:14.976172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:14.976464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:14.976739 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:14.996619 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:14.997795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:15.006417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1242/2513] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:15.009965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.019816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f3c7e-816d-495d-a213-1158631e04b9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.024943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.025538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.025793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.026060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.026694 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:20:15.026996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.028633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.041039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1272/2514] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:20:15.046170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.055653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6baa1-2d16-4ad8-ac76-30fc1cfb8df9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.060804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.061406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.061687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.061958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.062618 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.064698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.174230 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1243/2515] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:15 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:15.178452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.188637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7e062-2d97-4249-8e05-edf7996cb295 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.245875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.246480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.246757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.247030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.247691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.249695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.339280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1273/2516] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:15 2022] POST /identity/v3/users => generated 302 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:15.343453 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.354099 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733b708-3f1a-48a8-98ca-6aada29a1b0c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.358785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.359399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.359675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.359949 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.360583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.362165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.372379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1244/2517] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:15.376607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.382921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:15.387149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4fb0e-3268-4175-ae7e-a0d882c6a0de admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.391695 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.394537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.394537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.394537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.394537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.395055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1a92-95c5-462f-acdd-fbb554c08fb8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.406229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1274/2518] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:15.410371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.420440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35fd63-7078-4616-bece-cc976fdf8551 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.424980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.425608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.425882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.426156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] PATH_INFO: `/v3/projects/e5a0d5b6786b487da988809e2030560a/users/c22db3b005e44690a0da247cf94d5b29/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.441151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a0d5b6786b487da988809e2030560a, user_id=c22db3b005e44690a0da247cf94d5b29, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.443908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.565412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1245/2519] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:15 2022] PUT /identity/v3/projects/e5a0d5b6786b487da988809e2030560a/users/c22db3b005e44690a0da247cf94d5b29/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:15.571046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683239-bb75-444e-8b61-1abf383355d5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.572198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683239-bb75-444e-8b61-1abf383355d5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.572198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683239-bb75-444e-8b61-1abf383355d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.590996 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ec683239-bb75-444e-8b61-1abf383355d5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:15.592151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ec683239-bb75-444e-8b61-1abf383355d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:15.649900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1275/2520] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:15.659608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:15.664755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:15.720809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb26de-cb6f-491d-a689-926759674273 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:15.724723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:15.725405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:15.725684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:15.725954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:15.735263 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:15.736750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:15.746810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1246/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:16.945590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:16.946621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:16.947478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:16.976464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] MFA Rules not processed for user `c22db3b005e44690a0da247cf94d5b29`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:16.977659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:17.042931 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1276/2522] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:16 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:17.047502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.057480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433848-1273-43ab-8837-acfe1b97557b service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.062013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.062596 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.062871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.063174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.063797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.065689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.177693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1247/2523] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:17 2022] POST /identity/v3/projects => generated 401 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:17.182706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.193339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e51d9e-3e44-49bd-8ce3-ebcbb296efd3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.250336 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.250866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.251162 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.251438 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.252063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.254036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.361030 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1277/2524] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:17 2022] POST /identity/v3/users => generated 307 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:20:17.364762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.376405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cad20b-8a72-4a64-a351-d1813febec22 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.380961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.381559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.381839 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.382142 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.382784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.384519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.395052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1248/2525] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:17.399143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.408934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85750a77-660e-4393-a747-edb74be4d550 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.413476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.414114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.414407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.414657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] PATH_INFO: `/v3/projects/c257a48b919c41f39c4df3ce50c76459/users/469562f5e0134f0db2427788f07ed4df/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.426359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:17.430213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c257a48b919c41f39c4df3ce50c76459, user_id=469562f5e0134f0db2427788f07ed4df, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.434260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.499063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1278/2526] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:17 2022] PUT /identity/v3/projects/c257a48b919c41f39c4df3ce50c76459/users/469562f5e0134f0db2427788f07ed4df/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:17.503212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.513249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944abd68-c5cf-4475-b164-9bc9b9fbe0a7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.550692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:17.569875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.571508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.571508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.571508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.571961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.573369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26607d0d-72b1-4144-9f81-20cb97e54da3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.583122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1249/2527] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:17.587379 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:17.597602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22f62-f550-4360-9a37-f9d2a19d4403 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:17.602185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:17.604107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:17.604107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:17.604107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] PATH_INFO: `/v3/projects/c257a48b919c41f39c4df3ce50c76459/users/469562f5e0134f0db2427788f07ed4df/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:17.605986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c257a48b919c41f39c4df3ce50c76459, user_id=469562f5e0134f0db2427788f07ed4df, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:17.608648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:17.619465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1279/2528] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:17 2022] PUT /identity/v3/projects/c257a48b919c41f39c4df3ce50c76459/users/469562f5e0134f0db2427788f07ed4df/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:18.885229 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:18.885775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:18.885775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:18.913823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] MFA Rules not processed for user `469562f5e0134f0db2427788f07ed4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:18.917053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:18.988398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1250/2529] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:18 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:20:20.550115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:20.615299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0fe8-fb28-4de9-92d1-0d8ad0f38fa4 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:20.620471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:20.621061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:20.621338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:20.621736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] PATH_INFO: `/v3/users/c22db3b005e44690a0da247cf94d5b29` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:20.622881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c22db3b005e44690a0da247cf94d5b29)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:20.624898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:20.802334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c22db3b005e44690a0da247cf94d5b29'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:20.814991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c22db3b005e44690a0da247cf94d5b29'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:20.822606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c22db3b005e44690a0da247cf94d5b29'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:20.823526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c22db3b005e44690a0da247cf94d5b29'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:20.823948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c22db3b005e44690a0da247cf94d5b29'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:20.885246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1280/2530] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:20 2022] DELETE /identity/v3/users/c22db3b005e44690a0da247cf94d5b29 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:21.143565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:21.148329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1e6fe-3830-4a6d-9984-924ca4bb29f6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:21.206013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:21.206634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:21.206910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:21.207217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] PATH_INFO: `/v3/projects/e5a0d5b6786b487da988809e2030560a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:21.208321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a0d5b6786b487da988809e2030560a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:21.210297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:21.366082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a0d5b6786b487da988809e2030560a', 'request_id': 'req-b32988ef-3841-4f9c-ac3f-b93f0bb85160'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.372225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a0d5b6786b487da988809e2030560a', 'request_id': 'req-b32988ef-3841-4f9c-ac3f-b93f0bb85160'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.378358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a0d5b6786b487da988809e2030560a', 'request_id': 'req-b32988ef-3841-4f9c-ac3f-b93f0bb85160'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.380023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:21.485712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] The token cache is being invalidate because project e5a0d5b6786b487da988809e2030560a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:21.485930 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.486564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.488100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1251/2531] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:21 2022] DELETE /identity/v3/projects/e5a0d5b6786b487da988809e2030560a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:21.492178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:21.493650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:21.506794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41d324-56de-462f-a40d-8c0ccc4860b9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:21.557615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:21.564377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:21.565069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:21.565341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:21.565604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] PATH_INFO: `/v3/users/469562f5e0134f0db2427788f07ed4df` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:21.566779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469562f5e0134f0db2427788f07ed4df)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:21.568772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:21.767475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469562f5e0134f0db2427788f07ed4df'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.779867 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469562f5e0134f0db2427788f07ed4df'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.786629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469562f5e0134f0db2427788f07ed4df'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.787337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469562f5e0134f0db2427788f07ed4df'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.787951 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f4740588-d765-47f9-8859-29fa7f2bd036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469562f5e0134f0db2427788f07ed4df'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:21.874676 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1281/2532] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:21 2022] DELETE /identity/v3/users/469562f5e0134f0db2427788f07ed4df => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:22.166052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:22.167604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:22.180736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32988ef-3841-4f9c-ac3f-b93f0bb85160 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:22.237441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:22.238504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.238787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.239066 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] PATH_INFO: `/v3/projects/c257a48b919c41f39c4df3ce50c76459` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.240240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c257a48b919c41f39c4df3ce50c76459)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:22.242261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:22.364389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c257a48b919c41f39c4df3ce50c76459', 'request_id': 'req-8c3fffea-332c-48be-a44e-fe8e4917d226'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:22.370332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c257a48b919c41f39c4df3ce50c76459', 'request_id': 'req-8c3fffea-332c-48be-a44e-fe8e4917d226'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:22.376597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c257a48b919c41f39c4df3ce50c76459', 'request_id': 'req-8c3fffea-332c-48be-a44e-fe8e4917d226'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:22.474723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] The token cache is being invalidate because project c257a48b919c41f39c4df3ce50c76459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:22.474954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:22.475581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c3fffea-332c-48be-a44e-fe8e4917d226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:22.477094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1252/2533] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:22 2022] DELETE /identity/v3/projects/c257a48b919c41f39c4df3ce50c76459 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:20:22.482129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.482412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.482682 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.502886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:22.504103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:22.567564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1282/2534] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:22.575764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.576053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.576329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.595982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:22.597170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:22.605678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1253/2535] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:22.609857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:22.620052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50169471-909d-44d7-a432-1e55ab3b1b63 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:22.624638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:22.625230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.625510 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.625782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.626416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:20:22.626719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:22.628371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:22.640106 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1283/2536] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:20:22.644447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:22.653701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052d6ce-01bd-448a-a375-2a1b0290a2f7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:22.658308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:22.658928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.659225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.659496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.660141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:22.662043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:22.762582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1254/2537] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:22 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:22.766869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:22.777200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60964ce-9001-41a9-affc-bfd85f53e984 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:22.834854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:22.838678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:22.838678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:22.838678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:22.838678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:22.839095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:22.847910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:22.972837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1284/2538] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:22 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:22.978819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.000477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331080fc-c8f1-491f-964d-583308a03420 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.005436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.005984 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.006262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.006542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.007214 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.010340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.024046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1255/2539] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:23.025383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.037213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dd2b7-93b6-4766-bd6b-684dfaa1667c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.043465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.044305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.050368 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.050591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.051333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.052922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173868a-01b6-4eae-9e8d-17b18228b7a8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.065260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1285/2540] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:20:23.071268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.081385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cf62b-f047-40fc-b8b8-cad7d2353d2c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.085937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.087071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.087563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.087726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] PATH_INFO: `/v3/projects/34bb3fb91e284c4a9bd89be6d1ccb898/users/672a2706bf1f443ca8c490aa221007f5/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.105366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34bb3fb91e284c4a9bd89be6d1ccb898, user_id=672a2706bf1f443ca8c490aa221007f5, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.106026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.199295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1256/2541] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:23 2022] PUT /identity/v3/projects/34bb3fb91e284c4a9bd89be6d1ccb898/users/672a2706bf1f443ca8c490aa221007f5/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:23.210534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.210869 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.211654 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.217095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.218772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:23.240147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:23.241557 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] MFA Rules not processed for user `672a2706bf1f443ca8c490aa221007f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:23.243088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:23.277408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd14bc7-06f6-43b7-bb8a-d3478ad53711 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.281235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.281870 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.282151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.282432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.284275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:23.308141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1286/2542] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:23.312551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.362016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.363564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.374637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1257/2543] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:23.375669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ebd85-c856-40d6-bcbd-eedb3a62471b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.380231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.380817 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.381094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.381361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.381981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.384827 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.485864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1287/2544] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:23 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:20:23.489900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.500101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306f728-f186-406a-a8d0-8c6780082282 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.557096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.557698 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.557977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.558245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.558912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.561029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.674828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1258/2545] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:23 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:23.679788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.690901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43e7ff-e26b-4db8-b4ed-dab7e02a1bf6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.696746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.697354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.697633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.697900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.698621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.700370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.710804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1288/2546] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:23.717486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.728024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e2920-959a-44fd-bbdf-ff4f587bc243 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.732738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.733387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.733696 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.733988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] PATH_INFO: `/v3/projects/4864029e01784eefa7d465f35e289e9e/users/9e77dccaacd0445b877f046cc25834f2/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.749216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4864029e01784eefa7d465f35e289e9e, user_id=9e77dccaacd0445b877f046cc25834f2, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.751977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.838965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1259/2547] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:23 2022] PUT /identity/v3/projects/4864029e01784eefa7d465f35e289e9e/users/9e77dccaacd0445b877f046cc25834f2/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:23.843086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.854649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09d905-9d71-479c-b0ba-16ba34822a4a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.902485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:23.918148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.918747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.919054 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.919361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.919999 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.921591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801ef2-8598-4494-b649-59472b142e92 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.932212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1289/2548] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:23.936213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:23.946989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07931b9a-ee87-4b01-8305-df36bf55d9b1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:23.951586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:23.952216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.952493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.952763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] PATH_INFO: `/v3/projects/4864029e01784eefa7d465f35e289e9e/users/9e77dccaacd0445b877f046cc25834f2/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:23.954648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4864029e01784eefa7d465f35e289e9e, user_id=9e77dccaacd0445b877f046cc25834f2, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:23.957323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:23.967341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1260/2549] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:23 2022] PUT /identity/v3/projects/4864029e01784eefa7d465f35e289e9e/users/9e77dccaacd0445b877f046cc25834f2/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:20:23.977767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:23.978050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:23.978323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:24.005781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] MFA Rules not processed for user `9e77dccaacd0445b877f046cc25834f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:24.006998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:24.074018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1290/2550] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:24.085124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:24.098498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:24.142832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e5eed-0db9-414d-a282-9b9221eed20d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:24.146638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:24.147326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:24.147621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:24.148103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:24.157641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:24.159071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:24.170714 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1261/2551] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:24.378128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:24.387846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fa07b-498a-412a-9e69-427402c28308 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:24.391333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:24.391919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:24.392196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:24.392465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:24.451332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:24.452790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:24.462045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1291/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:34.164208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=21 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:34.457561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=71 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:39.159234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:39.160727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:39.218020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41269846-c220-408a-bc6f-0aa69b30c843 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:39.222020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:39.222620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:39.222901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:39.223206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:39.224947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:39.286665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:39.288166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:39.298561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1262/2553] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:43.027262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:43.028837 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:43.045538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029181e-ecde-4d08-8b40-85c983c1eee3 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:43.050122 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:43.050739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:43.053503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:43.053503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] PATH_INFO: `/v3/users/02240dc931e140498d82c6d91efb60c8` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:43.065209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02240dc931e140498d82c6d91efb60c8)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:43.067295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:43.232664 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02240dc931e140498d82c6d91efb60c8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.244940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02240dc931e140498d82c6d91efb60c8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.251990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02240dc931e140498d82c6d91efb60c8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.252656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02240dc931e140498d82c6d91efb60c8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.253249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84cf35a6-41bc-4ffd-b38c-2ab2009374cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02240dc931e140498d82c6d91efb60c8'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.322356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1292/2554] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:43 2022] DELETE /identity/v3/users/02240dc931e140498d82c6d91efb60c8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:43.579394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:43.591454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb2571-b02a-42ca-a64f-81d0ec168efa service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:43.667329 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:43.667954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:43.668231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:43.668507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] PATH_INFO: `/v3/projects/c1035740bd454149963828f12066b70d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:43.680048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1035740bd454149963828f12066b70d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:43.682545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:43.778287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:43.782020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1035740bd454149963828f12066b70d', 'request_id': 'req-e3bd2299-8d77-45c1-8c5b-683aa8e94875'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.787601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1035740bd454149963828f12066b70d', 'request_id': 'req-e3bd2299-8d77-45c1-8c5b-683aa8e94875'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.794826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1035740bd454149963828f12066b70d', 'request_id': 'req-e3bd2299-8d77-45c1-8c5b-683aa8e94875'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.883405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] The token cache is being invalidate because project c1035740bd454149963828f12066b70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:43.883707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.884642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e3bd2299-8d77-45c1-8c5b-683aa8e94875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:43.886136 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1263/2555] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:43 2022] DELETE /identity/v3/projects/c1035740bd454149963828f12066b70d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:43.890984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:43.891297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:43.891570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:43.916603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:43.916603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:43.972644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1293/2556] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:43.981281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:43.981560 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:43.981830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.001210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:44.002484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:44.010251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1264/2557] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:44.014638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.024942 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e065c-5bee-42c1-8ddf-24f5d8f31ad0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.029587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.030217 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.030473 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.030739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.031436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:20:44.031737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.033315 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.045544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1294/2558] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:20:44.049921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.059071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ff895-f4bc-466e-9a0e-35ca2e612009 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.063692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.064382 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.064831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.065272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.066039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.068176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.375187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1265/2559] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:44 2022] POST /identity/v3/projects => generated 405 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:44.376496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.388362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c8d6-19e0-4cc4-816f-783b7f03476c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.437488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:44.451133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.454518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.454518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.454518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.454518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.454917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.576646 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1295/2560] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:44 2022] POST /identity/v3/users => generated 303 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:20:44.585970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.597501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75373a-1ce7-4d96-b64a-e20b1a51ef66 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.601955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.602752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.603030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.603332 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.603961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.605614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.615808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1266/2561] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:44.619998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.629905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a8d52-9176-4497-9a08-2e4964da5f96 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.635000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.635738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.636066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.636378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.637007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.638621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1eea2-bfd1-4834-b6df-2d741c54a9ca admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.650319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1296/2562] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:44.663243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.673407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd08b43-753c-4a27-9eb7-079dc4b9cb23 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.677996 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.678631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.678907 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.679215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] PATH_INFO: `/v3/projects/5bec881dccc04ef4ad1a699d6a6d3928/users/d3fecda8b0824246976b76a369dd806b/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.694146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bec881dccc04ef4ad1a699d6a6d3928, user_id=d3fecda8b0824246976b76a369dd806b, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.697622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:44.767472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1267/2563] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:44 2022] PUT /identity/v3/projects/5bec881dccc04ef4ad1a699d6a6d3928/users/d3fecda8b0824246976b76a369dd806b/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:44.777586 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.777887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.778167 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.809378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] MFA Rules not processed for user `d3fecda8b0824246976b76a369dd806b`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:44.809378 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:44.871842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1297/2564] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:44.876522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:44.938152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e5d5-8c12-41d3-96ee-3f2abd9d6b12 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:44.942827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:44.943449 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:44.943730 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:44.944004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:44.944625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:44.947161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.027747 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1268/2565] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:44 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:45.031816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.043051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbbd33-9eeb-407f-8259-2af56560b4a6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.098647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.099203 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.099480 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.099744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.100361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.102932 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.169917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1298/2566] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:45 2022] POST /identity/v3/users => generated 309 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:45.173944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.185025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1086377-3b3f-4912-b496-1ec1079a22c4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.189715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.190296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.190573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.190842 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.191516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.193103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.203727 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1269/2567] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:45.207801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.217659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01e52c-6843-4cef-ab71-a34529d38cbb admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.222527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.227964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.227964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.227964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] PATH_INFO: `/v3/projects/780d52200f864bb39ed2c51e6e4f3340/users/787df58f727b4901913236976d6d7178/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.239001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780d52200f864bb39ed2c51e6e4f3340, user_id=787df58f727b4901913236976d6d7178, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.241798 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.333764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1299/2568] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:45 2022] PUT /identity/v3/projects/780d52200f864bb39ed2c51e6e4f3340/users/787df58f727b4901913236976d6d7178/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:45.337876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.347999 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932db09-cd45-4368-947f-1dc659aac949 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.387794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:45.406452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.407053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.407360 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.407633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.408317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.410002 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547d98-54e1-498e-b600-6cfa46c51ff7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.420587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1270/2569] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:45.425596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.436524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639ef6d-5607-4ea7-ac97-af1e679563e8 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.441386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.442115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.442483 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.442816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] PATH_INFO: `/v3/projects/780d52200f864bb39ed2c51e6e4f3340/users/787df58f727b4901913236976d6d7178/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.444825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780d52200f864bb39ed2c51e6e4f3340, user_id=787df58f727b4901913236976d6d7178, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.447490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.458275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1300/2570] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:45 2022] PUT /identity/v3/projects/780d52200f864bb39ed2c51e6e4f3340/users/787df58f727b4901913236976d6d7178/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:20:45.470817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.470946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.471236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.502980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] MFA Rules not processed for user `787df58f727b4901913236976d6d7178`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:45.506024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:45.575406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1271/2571] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:45.587764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.589520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:45.603578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:45.656033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc0138-1118-4085-b694-7f5dd21e3765 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.659691 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.660283 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.660556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.660857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.670607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.672454 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71111a0c-1784-44d8-acb2-088bc9770c12 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:45.681878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1301/2572] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:45.832200 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:45.892626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbdaa-42ee-4849-b262-32e43a407c4e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:45.897327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:45.897946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:45.898226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:45.898496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] PATH_INFO: `/v3/users/d3fecda8b0824246976b76a369dd806b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:45.900324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fecda8b0824246976b76a369dd806b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:45.902278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:46.175678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fecda8b0824246976b76a369dd806b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:46.187322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fecda8b0824246976b76a369dd806b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:46.188973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:46.248179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fecda8b0824246976b76a369dd806b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:46.248810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fecda8b0824246976b76a369dd806b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:46.249431 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fecda8b0824246976b76a369dd806b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:46.262666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1272/2573] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:45 2022] DELETE /identity/v3/users/d3fecda8b0824246976b76a369dd806b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:20:46.267153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:46.267663 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:46.267785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:46.286792 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:46.288023 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:46.347990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1302/2574] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:46.358280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:46.359779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:46.417780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0779-6462-46ad-8410-3188dd4a9132 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:46.421383 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:46.421979 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:46.422257 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:46.422526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:46.431743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:46.433111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:46.442293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1273/2575] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:46.815772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:46.824063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:46.837661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7eab-1d76-4848-af41-bdc0cfeab4d0 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:46.842343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:46.842976 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:46.843530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:46.843796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] PATH_INFO: `/v3/projects/5bec881dccc04ef4ad1a699d6a6d3928` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:46.844912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bec881dccc04ef4ad1a699d6a6d3928)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:46.846919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:46.992293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bec881dccc04ef4ad1a699d6a6d3928', 'request_id': 'req-b87bf40a-1ea3-4040-a559-a68e27c7ad26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.066294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bec881dccc04ef4ad1a699d6a6d3928', 'request_id': 'req-b87bf40a-1ea3-4040-a559-a68e27c7ad26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.073597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bec881dccc04ef4ad1a699d6a6d3928', 'request_id': 'req-b87bf40a-1ea3-4040-a559-a68e27c7ad26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.084021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] The token cache is being invalidate because project 5bec881dccc04ef4ad1a699d6a6d3928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:47.084304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.084923 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.086396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1303/2576] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:46 2022] DELETE /identity/v3/projects/5bec881dccc04ef4ad1a699d6a6d3928 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:47.090402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:47.100217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b348036-a98a-4086-9b13-7ede778b76f9 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:47.155244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:47.155866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:47.156143 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:47.156416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] PATH_INFO: `/v3/users/787df58f727b4901913236976d6d7178` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:47.157519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=787df58f727b4901913236976d6d7178)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:47.159461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:47.324442 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787df58f727b4901913236976d6d7178'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.336390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787df58f727b4901913236976d6d7178'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.415762 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787df58f727b4901913236976d6d7178'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.416387 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787df58f727b4901913236976d6d7178'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.417001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c4958fc5-49ed-4979-ae38-e6dee44850f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787df58f727b4901913236976d6d7178'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:47.429588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1274/2577] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:20:47 2022] DELETE /identity/v3/users/787df58f727b4901913236976d6d7178 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:47.816975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:47.818504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:47.831938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bf40a-1ea3-4040-a559-a68e27c7ad26 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:47.872453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:47.896418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:47.902836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:47.902836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:47.902836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] PATH_INFO: `/v3/projects/780d52200f864bb39ed2c51e6e4f3340` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:47.902836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780d52200f864bb39ed2c51e6e4f3340)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:47.902836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.018097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780d52200f864bb39ed2c51e6e4f3340', 'request_id': 'req-868ee171-29d1-4d92-8fbe-0118c817d3d7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:48.078111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780d52200f864bb39ed2c51e6e4f3340', 'request_id': 'req-868ee171-29d1-4d92-8fbe-0118c817d3d7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:48.084842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780d52200f864bb39ed2c51e6e4f3340', 'request_id': 'req-868ee171-29d1-4d92-8fbe-0118c817d3d7'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:48.096458 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] The token cache is being invalidate because project 780d52200f864bb39ed2c51e6e4f3340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:20:48.096675 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:48.097309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-868ee171-29d1-4d92-8fbe-0118c817d3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:20:48.098857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1304/2578] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:20:47 2022] DELETE /identity/v3/projects/780d52200f864bb39ed2c51e6e4f3340 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:48.103736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.104252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.104352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.112898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:48.123944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:48.125135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:48.182095 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1275/2579] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:48.189819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.190120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.190391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.210270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:48.212161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:48.220306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1305/2580] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:48.227288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.233803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186914-4b8f-4ee3-bd7f-b0e1d0aea087 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.238265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.239062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.239211 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.239400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.240031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:20:48.240328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.241909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.253346 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1276/2581] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:20:48.257587 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.267920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cdecf-892f-486a-b1f8-fb647666877a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.272800 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.279239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.279239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.279239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.279239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.279239 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.370707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1306/2582] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/projects => generated 355 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:48.374797 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.385369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167a66b-3ec7-42b8-b754-bf2e589e4fd8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.439750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.440337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.440612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.440880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.441494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.443480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.511243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1277/2583] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/users => generated 278 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:48.515281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.526189 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0774cf-3da8-45e1-8634-4535ca8524ac admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.530725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.531344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.531617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.531879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.532500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.534105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.543799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1307/2584] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:48.547561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.558109 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38d28-60ec-468c-96f8-9c83de2c7a24 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.562766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.563390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.563672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.563943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.564586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.566184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa8a67-7d13-443e-89fa-3e82699fbca0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.577738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1278/2585] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:48.581803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.592603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d9f5-46cb-43b1-b985-d5f0ffc23b59 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.597327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.597954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.598232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.598499 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] PATH_INFO: `/v3/projects/db5adf400d794c4c970ef4b3fa4f888f/users/587ba84561f142df85bddcc3bd308f84/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.612750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5adf400d794c4c970ef4b3fa4f888f, user_id=587ba84561f142df85bddcc3bd308f84, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.615424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.677861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1308/2586] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:48 2022] PUT /identity/v3/projects/db5adf400d794c4c970ef4b3fa4f888f/users/587ba84561f142df85bddcc3bd308f84/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:48.688219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.688498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.688771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.717647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] MFA Rules not processed for user `587ba84561f142df85bddcc3bd308f84`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:48.719134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:48.781017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:48.784590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1279/2587] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:48.789245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.852626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b02f-7075-4843-be71-7b4eaa0b9ed8 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:48.857245 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:48.857876 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:48.858107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:48.858371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:48.858989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:48.860980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:48.866906 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:48.940034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1309/2588] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:48.943973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:48.954751 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a86ce-af33-4a5b-b894-7ea992858980 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.010154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.010740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.011017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.011315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.011947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.013961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.112379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1280/2589] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:20:48 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:20:49.116131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.126823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b016281-d33f-4904-ac83-c4de64729803 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.131479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.132066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.132339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.132604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.133231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.135374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.145870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1310/2590] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:49.150980 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.161502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f7a-36ae-4b39-8552-8611f3f74a27 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.166337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.173732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.173732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.173732 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] PATH_INFO: `/v3/projects/e0e8f44181c04045abb6a0338476383d/users/1584275641ee4b8c953f5027ea11e883/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.182169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e8f44181c04045abb6a0338476383d, user_id=1584275641ee4b8c953f5027ea11e883, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.184921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.264122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1281/2591] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:49 2022] PUT /identity/v3/projects/e0e8f44181c04045abb6a0338476383d/users/1584275641ee4b8c953f5027ea11e883/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:49.269790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.280353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25e573-00ef-4803-a436-d1291d05280c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.336246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.337459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.337755 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.338056 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.338753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.340459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8a534-1ccd-45f6-ab1c-7041e3c1be2f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.350962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1311/2592] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:20:49.355151 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.365498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d089b-b42b-41a5-89e8-428e90b883cb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.370328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.370961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.371291 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.371563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] PATH_INFO: `/v3/projects/e0e8f44181c04045abb6a0338476383d/users/1584275641ee4b8c953f5027ea11e883/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.373495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e8f44181c04045abb6a0338476383d, user_id=1584275641ee4b8c953f5027ea11e883, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.376232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.387110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1282/2593] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:20:49 2022] PUT /identity/v3/projects/e0e8f44181c04045abb6a0338476383d/users/1584275641ee4b8c953f5027ea11e883/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:20:49.398294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.398578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.398846 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.426341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] MFA Rules not processed for user `1584275641ee4b8c953f5027ea11e883`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:20:49.427569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:49.493361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1312/2594] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:20:49.505076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.506785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:49.565834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23bbc9-70a0-47f7-b0b2-28134b52edd0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.569598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.570225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.570503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.570773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.580794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.582220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.591932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1283/2595] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:49.782090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:49.791624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee0e5e-2f5b-4f6d-a18e-41bebe1e83da None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:49.795120 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:49.795704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:49.795994 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:49.796271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:49.849578 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:49.851895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:49.853312 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:49.862898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1313/2596] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:55.006405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:20:55.007828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:55.019355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f76b9e-16a6-4bcb-a9c7-78b356e152ea service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:20:55.022882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:20:55.023493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:20:55.023773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:20:55.024043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:20:55.025678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:20:55.081231 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:20:55.108534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:20:55.109967 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:20:55.129176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1284/2597] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:20:59.915850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:02.120051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:02.121814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:02.217573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404cb21-4bd9-48e9-a873-e51f178e2f05 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:02.226094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:02.226723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:02.227005 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:02.227302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:02.229807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:02.246301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:02.248065 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:02.260018 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1314/2598] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:05.140069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=16 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:07.520807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:07.522216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:07.584471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91528078-86db-43d8-b5ac-8904910826aa service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:07.588039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:07.588640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:07.588946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:07.589251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:07.590974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:07.652656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:07.654110 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:07.664056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1285/2599] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:11.400151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:11.401924 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:11.416542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb9ed1-4046-4677-a70b-69b90c0245cb service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:11.421472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:11.422096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:11.422374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:11.422666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] PATH_INFO: `/v3/users/3d931a379404433b9c3c690dd9beb83d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:11.423793 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d931a379404433b9c3c690dd9beb83d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:11.426078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:11.610749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d931a379404433b9c3c690dd9beb83d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:11.623040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d931a379404433b9c3c690dd9beb83d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:11.629948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d931a379404433b9c3c690dd9beb83d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:11.630625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d931a379404433b9c3c690dd9beb83d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:11.631273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-5cef94c8-cbb6-4311-962a-7dd3c702b35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d931a379404433b9c3c690dd9beb83d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:11.697354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1315/2600] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:11 2022] DELETE /identity/v3/users/3d931a379404433b9c3c690dd9beb83d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:11.936011 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:11.946746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d96c2-85b8-4dce-83a1-fce17473a961 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.006957 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.007663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.007945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.008220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] PATH_INFO: `/v3/projects/b8799a08d0684b5c8d9cd3f89c3016d7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.009341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8799a08d0684b5c8d9cd3f89c3016d7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.011321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.150416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8799a08d0684b5c8d9cd3f89c3016d7', 'request_id': 'req-ccf039f4-56c9-4202-89df-d914293f4008'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:12.153024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:12.156937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8799a08d0684b5c8d9cd3f89c3016d7', 'request_id': 'req-ccf039f4-56c9-4202-89df-d914293f4008'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:12.162925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8799a08d0684b5c8d9cd3f89c3016d7', 'request_id': 'req-ccf039f4-56c9-4202-89df-d914293f4008'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:12.258497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] The token cache is being invalidate because project b8799a08d0684b5c8d9cd3f89c3016d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:12.258781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:12.259415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ccf039f4-56c9-4202-89df-d914293f4008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:12.261125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1286/2601] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:11 2022] DELETE /identity/v3/projects/b8799a08d0684b5c8d9cd3f89c3016d7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:12.266060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.266343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.266612 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.288198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:12.291616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:12.340133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:12.349978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1316/2602] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:12.358858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.359196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.359476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.379305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:12.380511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:12.388461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1287/2603] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:12.392730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.402197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111d60-db12-43b6-b5e4-eff8c09ef7b7 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.407639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.408232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.408509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.408781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.409412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:12.409707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.411338 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.424236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1317/2604] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:12.428672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.438078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c749-662c-4ef1-aaee-5a668447010e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.442826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.443522 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.443821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.444077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.444700 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.446740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.553491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1288/2605] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:12 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:12.557722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.568392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbdb65-d2b5-4028-9883-207276396d81 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.623258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.623854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.624129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.624396 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.625015 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.626901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.693416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1318/2606] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:12 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:12.697530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.709224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987075d-22cb-4dbc-8d18-0bfac764aa6b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.713955 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.714548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.714850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.715140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.715785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.717818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.727905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1289/2607] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:12.731879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.742392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b65334-6cd3-4d38-b132-2da01cc9114f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.747021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.748360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.748617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.748887 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.749522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.751138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dcd7b-e45e-4d05-ab71-8be4ae32dcca admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.761529 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1319/2608] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:12.765653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.775575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5ad1-893a-4aa8-91b4-762faed1a895 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:12.780415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:12.781052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.781334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.781611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] PATH_INFO: `/v3/projects/586913e10e634fbbabbf164847c75632/users/b6c18954d41e4c96aaf2f58e5515687c/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.796349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586913e10e634fbbabbf164847c75632, user_id=b6c18954d41e4c96aaf2f58e5515687c, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:12.799004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:12.857634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1290/2609] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:12 2022] PUT /identity/v3/projects/586913e10e634fbbabbf164847c75632/users/b6c18954d41e4c96aaf2f58e5515687c/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:12.868804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:12.869117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:12.869389 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:12.898228 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] MFA Rules not processed for user `b6c18954d41e4c96aaf2f58e5515687c`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:12.899450 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:12.961037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1320/2610] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:12.972821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:12.974547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:13.048606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837916d-9344-42d9-a987-ea2d7ba5ee98 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:13.054077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:13.054671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:13.054952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:13.055258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:13.063413 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:13.065228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a74e99-2a58-4d8f-bb67-13ce600d2ce0 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:13.073803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1291/2611] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:13.291877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:13.342439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:13.352944 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb6bea-1646-4ffe-9570-09d29b433df9 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:13.360173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:13.360173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:13.360173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:13.360173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] PATH_INFO: `/v3/users/b6c18954d41e4c96aaf2f58e5515687c` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:13.360615 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c18954d41e4c96aaf2f58e5515687c)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:13.362252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:13.593299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c18954d41e4c96aaf2f58e5515687c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:13.594979 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c18954d41e4c96aaf2f58e5515687c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:13.602000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c18954d41e4c96aaf2f58e5515687c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:13.604595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c18954d41e4c96aaf2f58e5515687c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:13.604595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c18954d41e4c96aaf2f58e5515687c'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:13.705270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1321/2612] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:13 2022] DELETE /identity/v3/users/b6c18954d41e4c96aaf2f58e5515687c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:13.709876 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:13.710166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:13.710457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:13.730465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:13.731726 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:13.793252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1292/2613] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:13.803779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:13.811550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:13.873957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774578f-62f9-40b9-990c-b01b274c940d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:13.877613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:13.878214 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:13.878490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:13.878760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:13.888878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:13.890645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04974b49-b55f-4d16-9ac5-21e319b693d2 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:13.900955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1322/2614] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:14.347168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:14.348735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:14.355771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:14.362934 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7341bb1-5389-4fed-bad1-366b440fc228 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:14.367775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:14.368404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:14.368818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:14.369096 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] PATH_INFO: `/v3/projects/586913e10e634fbbabbf164847c75632` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:14.370232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] RBAC: Authorizing `identity:delete_project(project_id=586913e10e634fbbabbf164847c75632)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:14.372276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:14.540875 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586913e10e634fbbabbf164847c75632', 'request_id': 'req-2b84afff-efbf-4bf1-ace2-af183501f36b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:14.547368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586913e10e634fbbabbf164847c75632', 'request_id': 'req-2b84afff-efbf-4bf1-ace2-af183501f36b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:14.554104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586913e10e634fbbabbf164847c75632', 'request_id': 'req-2b84afff-efbf-4bf1-ace2-af183501f36b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:14.635998 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] The token cache is being invalidate because project 586913e10e634fbbabbf164847c75632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:14.636434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:14.638028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2b84afff-efbf-4bf1-ace2-af183501f36b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:14.640761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1293/2615] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:14 2022] DELETE /identity/v3/projects/586913e10e634fbbabbf164847c75632 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:14.645809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:14.646355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:14.646490 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:14.667624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:14.668848 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:14.727600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1323/2616] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:14.735562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:14.735848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:14.736126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:14.756193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:14.757404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:14.766206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1294/2617] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:14.770655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:14.781334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0badb4-2c9f-441a-bbe4-2700b88c76f6 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:14.786094 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:14.786700 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:14.786980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:14.787289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:14.787969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:14.788274 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:14.789973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:14.802723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1324/2618] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:14.807634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:14.818085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c954-d350-4ee8-b82e-9cb82511919e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:14.823408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:14.824212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:14.824494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:14.824792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:14.826265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:14.828628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.042032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1295/2619] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:14 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:15.046212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.051282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:15.056428 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2bb25-b66e-4bf1-b2f6-53f5dc9106ee admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.111877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.112477 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.112753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.113026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.113647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.115591 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.184580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1325/2620] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:15 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:15.189477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.200026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0922-f3ca-4739-b9b9-606ab0375f61 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.204763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.205352 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.205635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.205912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.206547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.208177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.218407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1296/2621] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:15.222272 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.232674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ddb6c-df30-453d-80f6-d46b74cb667c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.237188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.237768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.238039 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.238306 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.238938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.240548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab1ab6-907d-49c8-8768-8e9e49bf6a6a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.252778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1326/2622] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:15.256848 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.267228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68c204-a600-43a8-848d-ca762395c6c2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.272058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.272810 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.272978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.273302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] PATH_INFO: `/v3/projects/7709b09f3b234cc29de253a5db0e6185/users/3567df3cc3cc4b06868df4200e30b9f3/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.289761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7709b09f3b234cc29de253a5db0e6185, user_id=3567df3cc3cc4b06868df4200e30b9f3, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.292646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.339078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1297/2623] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:15 2022] PUT /identity/v3/projects/7709b09f3b234cc29de253a5db0e6185/users/3567df3cc3cc4b06868df4200e30b9f3/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:15.350154 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.350435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.350709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.379809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] MFA Rules not processed for user `3567df3cc3cc4b06868df4200e30b9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:15.380905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:15.451276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1327/2624] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:15.455722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.519318 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fa1dd-5382-4ed3-90cb-81933a4bca16 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.524217 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.526255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.526255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.526255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.526255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.527990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.644210 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1298/2625] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:15 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:15.648131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.660981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05363113-b5d2-48f2-8cad-e2c34bb4862b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.731304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.732794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.732794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.732794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.733311 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.735233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.826542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:15.834596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1328/2626] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:15 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:15.836602 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.855146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b4b7-9118-427d-8fba-54fae2263a03 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.855146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.855146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.855146 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.855546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.856052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.858035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:15.859502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:15.872375 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1299/2627] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:15.873950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:15.883756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc85eb-7db0-442f-9b4f-9afafdb39157 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:15.888835 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:15.889506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:15.889788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:15.890061 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] PATH_INFO: `/v3/projects/34ae349a41c44ee0816889f13ac97028/users/b12e05f090584839bf9535bbfe762f0b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:15.906282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ae349a41c44ee0816889f13ac97028, user_id=b12e05f090584839bf9535bbfe762f0b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:15.908998 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.002152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1329/2628] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:15 2022] PUT /identity/v3/projects/34ae349a41c44ee0816889f13ac97028/users/b12e05f090584839bf9535bbfe762f0b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:16.006236 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:16.017985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89f0c5-0f14-484f-b497-ad7c4983865e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:16.079971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:16.080493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.080780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.081050 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.081686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:16.083418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089712c-8291-4155-a3bb-4fdb112cb4c0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.093565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1300/2629] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:16.097763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:16.108459 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1f0a5-9e96-4bab-be4c-9c403740df78 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:16.113130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:16.113765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.114044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.114318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] PATH_INFO: `/v3/projects/34ae349a41c44ee0816889f13ac97028/users/b12e05f090584839bf9535bbfe762f0b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.116303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ae349a41c44ee0816889f13ac97028, user_id=b12e05f090584839bf9535bbfe762f0b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:16.119051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.129737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1330/2630] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:16 2022] PUT /identity/v3/projects/34ae349a41c44ee0816889f13ac97028/users/b12e05f090584839bf9535bbfe762f0b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:16.142020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.142298 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.142565 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.165618 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:16.167397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:16.175195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] MFA Rules not processed for user `b12e05f090584839bf9535bbfe762f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:16.176402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:16.236836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88fd99-103b-44e1-aac9-7a814dca6b78 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:16.242055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:16.242794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.243063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.243375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] PATH_INFO: `/v3/users/587ba84561f142df85bddcc3bd308f84` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.244562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587ba84561f142df85bddcc3bd308f84)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:16.245534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1301/2631] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:16.246657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.258686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:16.260041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:16.316828 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f782c-1bc5-4d6f-a572-032f0b810575 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:16.320516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:16.321182 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.321457 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.321728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.332506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:16.333244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def8f72-adb6-484f-978a-f5848e86ca47 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.343290 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1302/2632] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:21:16.441201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587ba84561f142df85bddcc3bd308f84'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:16.454260 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587ba84561f142df85bddcc3bd308f84'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:16.461564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587ba84561f142df85bddcc3bd308f84'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:16.462291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587ba84561f142df85bddcc3bd308f84'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:16.463781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587ba84561f142df85bddcc3bd308f84'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:16.573885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1331/2633] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:16 2022] DELETE /identity/v3/users/587ba84561f142df85bddcc3bd308f84 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:16.580164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.580675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.580788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.600414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:16.601526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:16.614371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:16.663004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1303/2634] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:16.673833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:16.675603 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:16.722595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:16.734761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b938-e4ba-45f3-b0ee-4031ddcdd504 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:16.738455 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:16.743236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:16.743236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:16.743236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:16.751446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:16.752875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:16.762868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1332/2635] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:17.206794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:17.208253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.227344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d655d03-23e9-4dac-a40b-a2ae6fd6f1f8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:17.234941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:17.235879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.236207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.236441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] PATH_INFO: `/v3/projects/db5adf400d794c4c970ef4b3fa4f888f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.238057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5adf400d794c4c970ef4b3fa4f888f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:17.241350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:17.361222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:17.362927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.373938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5adf400d794c4c970ef4b3fa4f888f', 'request_id': 'req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.377686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae15a0-8eda-44e1-a185-c710c1f0d81e service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:17.379709 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5adf400d794c4c970ef4b3fa4f888f', 'request_id': 'req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.386648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5adf400d794c4c970ef4b3fa4f888f', 'request_id': 'req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.432706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:17.433330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.433605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.433873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] PATH_INFO: `/v3/users/3567df3cc3cc4b06868df4200e30b9f3` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.435983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3567df3cc3cc4b06868df4200e30b9f3)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:17.437973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:17.515611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] The token cache is being invalidate because project db5adf400d794c4c970ef4b3fa4f888f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:17.515963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.516617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.518127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1304/2636] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:17 2022] DELETE /identity/v3/projects/db5adf400d794c4c970ef4b3fa4f888f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:17.518889 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:17.520185 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.533377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c6b1c-6e11-4b48-9cb0-ad2e7ff5df83 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:17.538156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:17.538763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.539067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.539380 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] PATH_INFO: `/v3/users/672a2706bf1f443ca8c490aa221007f5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.540504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] RBAC: Authorizing `identity:delete_user(user_id=672a2706bf1f443ca8c490aa221007f5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:17.542428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:17.668033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3567df3cc3cc4b06868df4200e30b9f3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.680717 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3567df3cc3cc4b06868df4200e30b9f3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.687588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3567df3cc3cc4b06868df4200e30b9f3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.688279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3567df3cc3cc4b06868df4200e30b9f3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.688897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3567df3cc3cc4b06868df4200e30b9f3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.724181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672a2706bf1f443ca8c490aa221007f5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.737141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672a2706bf1f443ca8c490aa221007f5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.787610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1333/2637] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:17 2022] DELETE /identity/v3/users/3567df3cc3cc4b06868df4200e30b9f3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:17.790624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:17.790624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.845397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672a2706bf1f443ca8c490aa221007f5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.846045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672a2706bf1f443ca8c490aa221007f5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.846668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f02f542-5d65-485a-9ef2-198a5fd3924a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672a2706bf1f443ca8c490aa221007f5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:17.854285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e6ccc-25e4-419a-8eba-cb28f46bfb3a admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:17.861205 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:17.861814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1305/2638] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:17 2022] DELETE /identity/v3/users/672a2706bf1f443ca8c490aa221007f5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:17.862862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64d30-e163-4ea3-a7d0-0d89d6402046 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.863174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64d30-e163-4ea3-a7d0-0d89d6402046 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.865604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df64d30-e163-4ea3-a7d0-0d89d6402046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.865788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.865788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.865788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] PATH_INFO: `/v3/users/1584275641ee4b8c953f5027ea11e883` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.866617 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1584275641ee4b8c953f5027ea11e883)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:17.868561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:17.874328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:17.883041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2df64d30-e163-4ea3-a7d0-0d89d6402046 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:17.884320 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2df64d30-e163-4ea3-a7d0-0d89d6402046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.894554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:17.943292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1306/2639] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:17.944479 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:17.944765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:17.945041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:17.964075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:17.965273 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:17.973776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1307/2640] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:17.974547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:17.975755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:18.032900 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bb104-9db2-4710-bae4-1ecac3d0e18a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.036849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.037550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.037825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.038094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.047237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.048667 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:18.058030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1308/2641] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:18.058798 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:18.067866 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23905-fba5-418c-8bda-2e1a8081cb15 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.072675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.073226 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.073502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.074100 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.080870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1584275641ee4b8c953f5027ea11e883'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.085048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.086482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:18.097601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1309/2642] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:18.098774 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1584275641ee4b8c953f5027ea11e883'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.105737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1584275641ee4b8c953f5027ea11e883'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.106500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1584275641ee4b8c953f5027ea11e883'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.107289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1584275641ee4b8c953f5027ea11e883'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.156539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1334/2643] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:17 2022] DELETE /identity/v3/users/1584275641ee4b8c953f5027ea11e883 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:18.557499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:18.567945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:18.582029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17d925-c378-45c5-94cc-88e807c9ed42 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.639829 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.640499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.640784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.641060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] PATH_INFO: `/v3/projects/34bb3fb91e284c4a9bd89be6d1ccb898` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.642208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34bb3fb91e284c4a9bd89be6d1ccb898)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.644177 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:18.649149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:18.654151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:18.665082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a07f-b529-4806-9478-089d31c445da admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.670872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.671556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.671940 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.672211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] PATH_INFO: `/v3/projects/7709b09f3b234cc29de253a5db0e6185` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.673826 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7709b09f3b234cc29de253a5db0e6185)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.676516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:18.796555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:18.802531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34bb3fb91e284c4a9bd89be6d1ccb898', 'request_id': 'req-31534325-e186-4792-9389-519af1fbb731'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.804446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7709b09f3b234cc29de253a5db0e6185', 'request_id': 'req-6994376c-e20a-437a-9087-cfa5da091415'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.808504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34bb3fb91e284c4a9bd89be6d1ccb898', 'request_id': 'req-31534325-e186-4792-9389-519af1fbb731'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.815882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34bb3fb91e284c4a9bd89be6d1ccb898', 'request_id': 'req-31534325-e186-4792-9389-519af1fbb731'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.861911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7709b09f3b234cc29de253a5db0e6185', 'request_id': 'req-6994376c-e20a-437a-9087-cfa5da091415'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.868628 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7709b09f3b234cc29de253a5db0e6185', 'request_id': 'req-6994376c-e20a-437a-9087-cfa5da091415'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.879899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] The token cache is being invalidate because project 7709b09f3b234cc29de253a5db0e6185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:18.880185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.880812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.882273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1335/2644] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:18 2022] DELETE /identity/v3/projects/7709b09f3b234cc29de253a5db0e6185 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:18.883790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:18.885108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:18.914330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] The token cache is being invalidate because project 34bb3fb91e284c4a9bd89be6d1ccb898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:18.914616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.915277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:18.917013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1310/2645] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:18 2022] DELETE /identity/v3/projects/34bb3fb91e284c4a9bd89be6d1ccb898 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:18.917553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:18.919396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:18.953397 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994376c-e20a-437a-9087-cfa5da091415 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.958386 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.959019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.959325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.959600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] PATH_INFO: `/v3/projects/e0e8f44181c04045abb6a0338476383d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.960739 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e8f44181c04045abb6a0338476383d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.962643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:18.977273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:18.981128 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31534325-e186-4792-9389-519af1fbb731 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:18.986398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:18.987020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:18.987326 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:18.987599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] PATH_INFO: `/v3/users/b12e05f090584839bf9535bbfe762f0b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:18.988824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12e05f090584839bf9535bbfe762f0b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:18.990622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.058833 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e8f44181c04045abb6a0338476383d', 'request_id': 'req-8c99ccc7-7687-4a04-adae-e138c590a9e3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.144213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e8f44181c04045abb6a0338476383d', 'request_id': 'req-8c99ccc7-7687-4a04-adae-e138c590a9e3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.150970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e8f44181c04045abb6a0338476383d', 'request_id': 'req-8c99ccc7-7687-4a04-adae-e138c590a9e3'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.158463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12e05f090584839bf9535bbfe762f0b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.161822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] The token cache is being invalidate because project e0e8f44181c04045abb6a0338476383d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:19.162115 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.162773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.164514 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1336/2646] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:18 2022] DELETE /identity/v3/projects/e0e8f44181c04045abb6a0338476383d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:19.168577 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.169981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.172135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12e05f090584839bf9535bbfe762f0b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.230446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12e05f090584839bf9535bbfe762f0b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.231083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12e05f090584839bf9535bbfe762f0b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.231933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-be099896-4778-40a6-b628-00e480a43145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12e05f090584839bf9535bbfe762f0b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.236151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99ccc7-7687-4a04-adae-e138c590a9e3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.241160 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.241872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.242183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.242476 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] PATH_INFO: `/v3/users/9e77dccaacd0445b877f046cc25834f2` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.243634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e77dccaacd0445b877f046cc25834f2)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.245637 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1311/2647] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:18 2022] DELETE /identity/v3/users/b12e05f090584839bf9535bbfe762f0b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:19.245845 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.246506 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ad5b-045a-4f4a-bd31-ec75119f8943 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.247452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ad5b-045a-4f4a-bd31-ec75119f8943 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.247733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218ad5b-045a-4f4a-bd31-ec75119f8943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.270542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a218ad5b-045a-4f4a-bd31-ec75119f8943 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:19.272031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a218ad5b-045a-4f4a-bd31-ec75119f8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.320472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:19.336474 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1312/2648] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:19.344890 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.345093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.345268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.364544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:19.365791 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.374419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1313/2649] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:21:19.378731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.388480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d8e4-d0b9-4d21-bd51-bab888190e8b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.393228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.393804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.394085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.394489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.395132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:19.395447 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.397024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.408258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1314/2650] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:21:19.412386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.422101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14dee9-f679-4f00-9adf-71743074834e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.426679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.427258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.427536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.427806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.428411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.430325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.456164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e77dccaacd0445b877f046cc25834f2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.468678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e77dccaacd0445b877f046cc25834f2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.475788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e77dccaacd0445b877f046cc25834f2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.476768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e77dccaacd0445b877f046cc25834f2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.477377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e77dccaacd0445b877f046cc25834f2'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.544685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1315/2651] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:21:19.545761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.547405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.578733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1337/2652] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:19 2022] DELETE /identity/v3/users/9e77dccaacd0445b877f046cc25834f2 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:19.618082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bbae7-6f5f-4209-978c-3ca3ab9c3ae7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.672734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.674385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.677497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.678532 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.678953 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.679392 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.680048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.682289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.706359 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7656f-c5ca-4f7b-b633-ffc0ac62af38 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.711786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.712549 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.712894 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.713225 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] PATH_INFO: `/v3/projects/34ae349a41c44ee0816889f13ac97028` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.714420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ae349a41c44ee0816889f13ac97028)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.716737 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.728213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:19.761838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1316/2653] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:21:19.766007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.775444 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74482-669f-4d8f-961e-e6aa081a7246 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.780349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.780920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.781196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.781465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.782090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.783822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.793481 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1317/2654] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:21:19.798916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.808402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e86d-22af-49ee-adfb-ee6b13d982c1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.813735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.814281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.814556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.814862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.815504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.817091 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.828042 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1318/2655] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:21:19.831874 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:19.837692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ae349a41c44ee0816889f13ac97028', 'request_id': 'req-7b2f26f0-1270-4269-9139-d63941686588'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.842912 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141495ab-86c9-4013-b40e-244c6363f98f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:19.865043 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:19.904543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:19.905208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.905590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.905873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] PATH_INFO: `/v3/projects/fdd744502b6747ee83dd50bf6aa4a2fa/users/77226c1d77354e878897df51ef858df6/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.907953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ae349a41c44ee0816889f13ac97028', 'request_id': 'req-7b2f26f0-1270-4269-9139-d63941686588'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.915461 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ae349a41c44ee0816889f13ac97028', 'request_id': 'req-7b2f26f0-1270-4269-9139-d63941686588'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.924201 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd744502b6747ee83dd50bf6aa4a2fa, user_id=77226c1d77354e878897df51ef858df6, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:19.927480 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513557d-a347-4083-8f54-f2b775dc857a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:19.928948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] The token cache is being invalidate because project 34ae349a41c44ee0816889f13ac97028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:19.929371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.930269 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7b2f26f0-1270-4269-9139-d63941686588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:19.931880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1338/2656] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:19 2022] DELETE /identity/v3/projects/34ae349a41c44ee0816889f13ac97028 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:19.937352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b14c-739b-484c-8ff8-b89010c18554 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.940813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b14c-739b-484c-8ff8-b89010c18554 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.940813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b14c-739b-484c-8ff8-b89010c18554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.958279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7139b14c-739b-484c-8ff8-b89010c18554 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:19.959857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7139b14c-739b-484c-8ff8-b89010c18554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:19.968140 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1339/2657] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:21:19.975986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:19.976278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:19.976540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:19.995791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:19.997545 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.005447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1340/2658] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:21:20.009645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.011300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1319/2659] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:19 2022] PUT /identity/v3/projects/fdd744502b6747ee83dd50bf6aa4a2fa/users/77226c1d77354e878897df51ef858df6/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:20.019507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f07f9-0a3a-4529-b466-a374375f6672 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.023539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.023819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.024089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.051539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] MFA Rules not processed for user `77226c1d77354e878897df51ef858df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:20.052770 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.078232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.078987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.079295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.079567 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.080197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:20.080585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.082229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.104246 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1341/2660] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:20.105134 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.107055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.119569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab379d-cdb3-4e2f-862d-7545c32e52f9 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.129137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1320/2661] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:21:20.129502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.129502 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.129679 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.129765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.129765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] PATH_INFO: `/v3/projects/4864029e01784eefa7d465f35e289e9e` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.129765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4864029e01784eefa7d465f35e289e9e)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.130192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.140237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b23c1-e41f-4365-8db2-adb96a295a43 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.144855 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.145455 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.145737 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.146009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.146639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.148800 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.249376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1321/2662] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:20.249599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4864029e01784eefa7d465f35e289e9e', 'request_id': 'req-b2265f35-89dd-403c-91d6-985839399df5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:20.253482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.255202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.320045 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01880144-8b3e-4e4d-a926-32e8a8aa8b38 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.324675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.325278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.325644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.325927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.326547 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.328566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184af93-6ea0-48e3-a67c-f262abbd7fc3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.378318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4864029e01784eefa7d465f35e289e9e', 'request_id': 'req-b2265f35-89dd-403c-91d6-985839399df5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:20.385553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4864029e01784eefa7d465f35e289e9e', 'request_id': 'req-b2265f35-89dd-403c-91d6-985839399df5'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:20.396224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] The token cache is being invalidate because project 4864029e01784eefa7d465f35e289e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:20.396509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:20.397153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:20.398634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1342/2663] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:20 2022] DELETE /identity/v3/projects/4864029e01784eefa7d465f35e289e9e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:20.399565 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.400823 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.419954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:20.427477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1322/2664] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:20.428718 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.429005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.429403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.449627 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:20.452427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.465135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:20.469050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2265f35-89dd-403c-91d6-985839399df5 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.511148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1323/2665] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:20.517962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.517962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.525745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ae794-fab7-40ea-90ad-bb03dc427462 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.526231 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.527090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.527495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.528763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.529448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.531363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.531535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d770f-6884-43a1-9d0d-8839b3f93831 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.531978 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.532241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.532563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.533161 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.535071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.613736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1343/2666] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:20.614987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.615287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.615556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.625963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1324/2667] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:20.626284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.636660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:20.638317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fbb27-9fa7-4bde-b58b-860ec9b72931 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.638506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:20.643207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.645281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.645281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.645281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.645281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.646597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.649743 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1344/2668] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:20.650745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.657160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1325/2669] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:20.660990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.661627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e747d8-1fc1-4ec0-9838-af0bccb4f27e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.666282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.667211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.667211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.667439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.668111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.669866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.671653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e6887-dd62-45e6-a2a4-804624c48a62 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.676584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.677175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.677460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.677738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.678372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:20.678678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.680354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.684064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1345/2670] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:20.684064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.695525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1326/2671] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:20.695525 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.695753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81b5e-21d2-4cf0-9ecf-ee4aa2148304 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.701059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.701788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.702066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.702343 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.702971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.704595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.708288 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ff5a-8dcb-4b76-ab29-ac47b233201d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.713427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.714053 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.714369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.714640 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.714987 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1346/2672] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:20.715582 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.716163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.717168 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.726558 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd019-c6c0-45b0-b787-e94d050fa409 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.730766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1327/2673] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:20.730766 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.734778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.735407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.735711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.735985 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.736616 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.738538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.738689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8fcc-d025-4c41-892c-f9bc44f87967 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.743465 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.744093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.744372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.744646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] PATH_INFO: `/v3/projects/932afa9ca4cf473ba9b0035a8440bd2e/users/f29d13894ed746f48798d7122833867d/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.759703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=932afa9ca4cf473ba9b0035a8440bd2e, user_id=f29d13894ed746f48798d7122833867d, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.762489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.842673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1347/2674] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:20.843653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.853734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1328/2675] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:20 2022] PUT /identity/v3/projects/932afa9ca4cf473ba9b0035a8440bd2e/users/f29d13894ed746f48798d7122833867d/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:20.854550 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:20.905633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:20.916715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48606ff-d226-4785-9b69-400f759bf7fd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.919365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327ce4-2ec5-4cb7-9222-6bd5343233c7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:20.922153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.922750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.923032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.923339 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.923968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.924197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:20.925834 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:20.926078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:20.926188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:20.926487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] PATH_INFO: `/v3/projects/63debe76964d47bba6e4fa073f2a31ad/users/f0ca8bdc5c214c4ea50e9d5db47243c1/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:20.942051 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63debe76964d47bba6e4fa073f2a31ad, user_id=f0ca8bdc5c214c4ea50e9d5db47243c1, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:20.944730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5b66c-1626-464b-9f5c-bdd5c5e287c8 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.033634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1348/2676] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:20 2022] PUT /identity/v3/projects/63debe76964d47bba6e4fa073f2a31ad/users/f0ca8bdc5c214c4ea50e9d5db47243c1/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:21.034831 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1329/2677] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:20 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.034984 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.035084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.035413 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.038863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.050341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cefb48-3140-4be4-9e42-7b7bac01cc1a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.058114 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:21.069385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] MFA Rules not processed for user `f29d13894ed746f48798d7122833867d`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:21.070619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:21.110250 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.110858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.111587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.111864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.112586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.114207 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a57f8-81a4-4fae-abe1-064095ba753c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.125013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1330/2678] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:21:21.127330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.128862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.129166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.140821 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1349/2679] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:21.141775 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.152553 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f5ae8-f782-4e7b-99a7-b63b7cb947ef None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.157367 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.157534 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] MFA Rules not processed for user `f0ca8bdc5c214c4ea50e9d5db47243c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:21.157889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.158190 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.158481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.158593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:21.159149 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.160753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.171145 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1350/2680] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:21.174926 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.185046 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac40d86-94f8-47b5-8af5-f79a87c71744 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.189653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.190233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.190507 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.190770 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.191447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.193381 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.225401 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1331/2681] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:21.226216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.237005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d12b0-4207-45bd-8dae-4feaaa620e1b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.241672 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.242295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.242570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.242838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] PATH_INFO: `/v3/projects/2db2674ebe3d4eb98c5653f08d0ac0ea/users/831c626055c94fa786ea1eee250025b9/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.258292 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db2674ebe3d4eb98c5653f08d0ac0ea, user_id=831c626055c94fa786ea1eee250025b9, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.261000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.326574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1351/2682] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.327576 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.365267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1332/2683] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:21 2022] PUT /identity/v3/projects/2db2674ebe3d4eb98c5653f08d0ac0ea/users/831c626055c94fa786ea1eee250025b9/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:21.366073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.396512 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6489f07-f504-432b-8fa3-711deed3f7dc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.430290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:21.434406 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9802b-15da-4232-8e0f-e07a843a6d27 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.439041 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.443012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.443012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.443012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.443012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.443012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cb95e-8dae-4c5a-9ae6-cb0456ee3cd6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.458188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.458796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.459077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.459400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.460032 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.461929 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.545535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1333/2684] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/users => generated 303 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.546785 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.547064 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.547358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.556155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:21.562393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1352/2685] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.562393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.579858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] MFA Rules not processed for user `831c626055c94fa786ea1eee250025b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:21.581184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:21.626063 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d2b44-a2b3-43cb-8320-611c9804c4a0 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.630765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.631409 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.631684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.631955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.632621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.634235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.644897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1353/2686] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:21.645761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.649180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1334/2687] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:21.650144 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.656660 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f7ca5-de35-4812-95f4-428dbd6f9134 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.661436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.662017 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.662293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.662491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f304fff-2518-4c0c-bffa-3b5587084c38 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.662594 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.663195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.665093 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.667264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.667936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.668222 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.668495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] PATH_INFO: `/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90/users/f43cfaf95ee74803978ccc4decc3f3a6/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.683941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad66c152f7147098ecd3ace1a3c3c90, user_id=f43cfaf95ee74803978ccc4decc3f3a6, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.687616 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.767285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1354/2688] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.768152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.777527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1335/2689] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:21 2022] PUT /identity/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90/users/f43cfaf95ee74803978ccc4decc3f3a6/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:21.777813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a3614-b906-46bc-a80c-9943f0f70549 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.778931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.836725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.837333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.837621 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.837889 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.838518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.841079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.841928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a4ee6-e4b7-4306-b375-5553022e3a30 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.846613 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.847243 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.847531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.847804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.848441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.850030 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.860623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1336/2690] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:21.861428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.871164 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4306d-e129-4234-9af3-57efdf285879 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.876683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.877253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.877531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.877891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.878426 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.880067 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.890938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1337/2691] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:21.894169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.900885 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a678-a938-403b-86c9-aeb56a5971d5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:21.905494 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:21.906089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:21.906366 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:21.906641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] PATH_INFO: `/v3/projects/49208892ce7540a3b7d6b8a16474ed08/users/437921ba4bed4cd88cbb698ccf28cf3a/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:21.918258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1355/2692] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:21.918258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.924101 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49208892ce7540a3b7d6b8a16474ed08, user_id=437921ba4bed4cd88cbb698ccf28cf3a, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:21.926879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:21.945457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:21.978918 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1338/2693] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:21 2022] PUT /identity/v3/projects/49208892ce7540a3b7d6b8a16474ed08/users/437921ba4bed4cd88cbb698ccf28cf3a/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:21.979598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:21.984744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8a6e6-2ba0-458e-b213-33f09d161313 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.009964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:22.039111 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.039884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.040164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.040433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] PATH_INFO: `/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90/users/f43cfaf95ee74803978ccc4decc3f3a6/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.041586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05455ce-7062-4c91-9094-e54d47159b62 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.042583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad66c152f7147098ecd3ace1a3c3c90, user_id=f43cfaf95ee74803978ccc4decc3f3a6, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.045254 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.046484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.047079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.047399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.047674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.048289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.050238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.055879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1356/2694] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:21 2022] PUT /identity/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90/users/f43cfaf95ee74803978ccc4decc3f3a6/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:22.056751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.118010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d823-20d8-4dfd-a16b-c61b81ba2c1d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.123504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.124082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.124354 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.124619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.125261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.126829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38485e08-1d5f-4f9c-9705-50f296a27dcd admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.136754 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1357/2695] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:22.138500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a78292-fc1f-42df-8f2f-6844cd02aafc None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.138842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a78292-fc1f-42df-8f2f-6844cd02aafc None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.139132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a78292-fc1f-42df-8f2f-6844cd02aafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.139254 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1339/2696] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:21 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:22.144538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.154446 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503d45-d646-4143-b58d-af0469ead29f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.159075 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.159743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.160082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.160317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] PATH_INFO: `/v3/projects/49208892ce7540a3b7d6b8a16474ed08/users/437921ba4bed4cd88cbb698ccf28cf3a/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.162285 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49208892ce7540a3b7d6b8a16474ed08, user_id=437921ba4bed4cd88cbb698ccf28cf3a, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.165077 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.166749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-74a78292-fc1f-42df-8f2f-6844cd02aafc None None] MFA Rules not processed for user `f43cfaf95ee74803978ccc4decc3f3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:22.168008 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-74a78292-fc1f-42df-8f2f-6844cd02aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:22.175867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1340/2697] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:22 2022] PUT /identity/v3/projects/49208892ce7540a3b7d6b8a16474ed08/users/437921ba4bed4cd88cbb698ccf28cf3a/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:22.176671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.229282 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:22.242353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffcf0b-9f9c-4b1f-8164-9cdacbad3c37 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.248034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.250179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.250179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.250179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.250179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.250915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1358/2698] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:22.251606 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.255582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.255872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.256147 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.263012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1341/2699] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:22.266961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.276803 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915582b8-3d62-4856-88ed-0205d5db2192 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.281830 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.282468 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.282750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.283117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] PATH_INFO: `/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0/users/59680166042d47ef8abeac0e5c036fce/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.284382 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] MFA Rules not processed for user `437921ba4bed4cd88cbb698ccf28cf3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:22.285734 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:22.298334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e5337ec4b448fdb79b47cce3bcc7f0, user_id=59680166042d47ef8abeac0e5c036fce, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.301103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.353575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1359/2700] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:22.354419 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.355721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:22.396683 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1342/2701] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:22 2022] PUT /identity/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0/users/59680166042d47ef8abeac0e5c036fce/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:22.397917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.400488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:22.414421 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45d59d-84bf-4047-ad2b-ea43942bfc64 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.459515 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d9a0-d14c-42a0-9e26-71e730297738 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.463203 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.463802 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.464078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.464208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.464349 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.464962 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.465635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.466045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.494835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:22.511986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:22.526438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.527920 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.531399 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.532832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.538367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1343/2702] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:22.539126 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.545057 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1360/2703] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:22.601281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.608964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499079e8-6746-4629-b39e-26c575e27b63 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.611485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e31856-10f1-46c2-beee-5c3f3d76ca94 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.613733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.614272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.614594 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.615070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.615744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.615870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.616541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.616667 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.616989 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.617589 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.629548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1344/2704] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:22.633559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.645546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862305-589c-45e9-be3a-4f65a61c581a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.651636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.652710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.653018 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.653330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] PATH_INFO: `/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0/users/59680166042d47ef8abeac0e5c036fce/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.655384 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e5337ec4b448fdb79b47cce3bcc7f0, user_id=59680166042d47ef8abeac0e5c036fce, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.658035 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e952-a503-4f55-94e8-0b4c0ad36dc3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.670220 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1345/2705] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:22 2022] PUT /identity/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0/users/59680166042d47ef8abeac0e5c036fce/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:22.681321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.686228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.686228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.686469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.686469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.694301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1361/2706] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:21:22.712303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] MFA Rules not processed for user `59680166042d47ef8abeac0e5c036fce`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:22.713487 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:22.786841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1346/2707] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:21:22.814742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:22.823478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da843d3f-c010-4fe3-a4b7-2754e6743e73 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:22.873449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:22.874074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:22.874351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:22.874625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:22.930497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:22.931993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:22.940726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1362/2708] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:24.084280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:24.094988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdde68-003f-41f6-9dcd-7de5407a0350 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:24.098651 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:24.099282 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:24.099563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:24.099836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:24.101552 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:24.134402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:24.160331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:24.161794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4bd6-e68c-49cd-874a-bae0ba7b74ed service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:24.171004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1347/2709] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:27.476171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:27.512496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:27.541605 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b9075-7a51-4bee-bae7-b36f1663c813 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:27.546920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:27.547613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:27.547893 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:27.548193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] PATH_INFO: `/v3/users/f29d13894ed746f48798d7122833867d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:27.549321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29d13894ed746f48798d7122833867d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:27.551273 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:27.762715 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29d13894ed746f48798d7122833867d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:27.775874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29d13894ed746f48798d7122833867d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:27.784636 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29d13894ed746f48798d7122833867d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:27.796953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29d13894ed746f48798d7122833867d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:27.796953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29d13894ed746f48798d7122833867d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:27.881456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1363/2710] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:27 2022] DELETE /identity/v3/users/f29d13894ed746f48798d7122833867d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:27.886167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:27.886670 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:27.886909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:27.907648 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:27.908910 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:27.971244 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1348/2711] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:27.980921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:27.982304 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:28.040497 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2c3bc-2220-43ab-b4fc-aeb55d8a73c6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:28.044121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:28.044723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:28.045004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:28.045280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:28.054300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:28.055776 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:28.065990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1364/2712] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:28.454843 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:28.456300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:28.469834 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e995b-9914-4c51-91e9-0d23ad917152 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:28.474645 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:28.475376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:28.475659 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:28.476579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] PATH_INFO: `/v3/projects/932afa9ca4cf473ba9b0035a8440bd2e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:28.477065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932afa9ca4cf473ba9b0035a8440bd2e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:28.478990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:28.588544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932afa9ca4cf473ba9b0035a8440bd2e', 'request_id': 'req-48ab5770-eeda-4674-ad54-d38ca3dc9850'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.595074 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932afa9ca4cf473ba9b0035a8440bd2e', 'request_id': 'req-48ab5770-eeda-4674-ad54-d38ca3dc9850'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.601436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932afa9ca4cf473ba9b0035a8440bd2e', 'request_id': 'req-48ab5770-eeda-4674-ad54-d38ca3dc9850'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.677935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] The token cache is being invalidate because project 932afa9ca4cf473ba9b0035a8440bd2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:28.678221 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.678857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.680356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1349/2713] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:28 2022] DELETE /identity/v3/projects/932afa9ca4cf473ba9b0035a8440bd2e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:28.684326 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:28.694347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c721ec1-4180-49f6-8ebc-f2843be50baa service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:28.742092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:28.751983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:28.756883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:28.756883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:28.756883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] PATH_INFO: `/v3/users/f43cfaf95ee74803978ccc4decc3f3a6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:28.756883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43cfaf95ee74803978ccc4decc3f3a6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:28.756883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:28.941390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43cfaf95ee74803978ccc4decc3f3a6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.953968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43cfaf95ee74803978ccc4decc3f3a6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.960991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43cfaf95ee74803978ccc4decc3f3a6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.961658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43cfaf95ee74803978ccc4decc3f3a6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:28.962256 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-7526ac17-aed0-4a13-9219-c5b994ff2599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43cfaf95ee74803978ccc4decc3f3a6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.046310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1365/2714] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:28 2022] DELETE /identity/v3/users/f43cfaf95ee74803978ccc4decc3f3a6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:29.463635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:29.465931 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:29.480542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab5770-eeda-4674-ad54-d38ca3dc9850 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:29.533171 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:29.538968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:29.539653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:29.539938 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:29.545756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] PATH_INFO: `/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:29.545756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad66c152f7147098ecd3ace1a3c3c90)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:29.545756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:29.672871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad66c152f7147098ecd3ace1a3c3c90', 'request_id': 'req-973de2ef-d204-4cba-a22d-bd2247d37020'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.678337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad66c152f7147098ecd3ace1a3c3c90', 'request_id': 'req-973de2ef-d204-4cba-a22d-bd2247d37020'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.684705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad66c152f7147098ecd3ace1a3c3c90', 'request_id': 'req-973de2ef-d204-4cba-a22d-bd2247d37020'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.774538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] The token cache is being invalidate because project 1ad66c152f7147098ecd3ace1a3c3c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:29.774816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.775469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-973de2ef-d204-4cba-a22d-bd2247d37020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:29.776963 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1350/2715] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:29 2022] DELETE /identity/v3/projects/1ad66c152f7147098ecd3ace1a3c3c90 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:29.781943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:29.782424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:29.782424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:29.803761 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:29.804948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:29.865714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1366/2716] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:29.873414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04925ea-7feb-4067-bd29-857397461522 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:29.873699 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04925ea-7feb-4067-bd29-857397461522 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:29.874543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04925ea-7feb-4067-bd29-857397461522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:29.894286 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f04925ea-7feb-4067-bd29-857397461522 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:29.895517 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f04925ea-7feb-4067-bd29-857397461522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:29.903583 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1351/2717] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:29.907748 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:29.917267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552353d-464a-40e6-8c26-dedff4b2a78f None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:29.922491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:29.923141 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:29.923524 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:29.923677 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:29.924313 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:29.924607 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:29.926174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:29.938629 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1367/2718] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:21:29.943130 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04925ea-7feb-4067-bd29-857397461522 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:29.953817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04925ea-7feb-4067-bd29-857397461522 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:29.958509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:29.959087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:29.959396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:29.959664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:29.960327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:29.962206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.052009 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1352/2719] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:29 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:30.056130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.068303 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242842b-b371-4de1-9d77-0723d396e57d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.125001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.125590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.125862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.126130 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.126753 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.128673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.136581 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:30.216042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1368/2720] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:30 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:21:30.220792 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.231049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b42900-3516-4190-9368-e591220073e3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.236944 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.237722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.237995 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.238262 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.238898 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.240639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.251535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1353/2721] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:30.255412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.265704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1743d-ade5-4da5-ace9-096c4512847b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.270759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.271431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.271713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.271988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.272632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.274294 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.285958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1369/2722] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:30.290369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.301692 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f57976-c144-4a21-a64a-f2db68c7686b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.306826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.307160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.307445 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.307748 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] PATH_INFO: `/v3/projects/9d7ff71e23bb4a75b2c84b2a6c716bbe/users/55c65749779c48e8b0850001c00e89be/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.323489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7ff71e23bb4a75b2c84b2a6c716bbe, user_id=55c65749779c48e8b0850001c00e89be, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.326289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31017f-d48c-4f9a-a344-ef88b7e8ceb4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.342195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.344724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:30.403224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b65726-0078-4ffe-b46c-61016ecdaab0 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.407764 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.408360 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.408757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.409031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.410788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:30.416122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1354/2723] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:30 2022] PUT /identity/v3/projects/9d7ff71e23bb4a75b2c84b2a6c716bbe/users/55c65749779c48e8b0850001c00e89be/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:30.428896 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.429251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.438345 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.461611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] MFA Rules not processed for user `55c65749779c48e8b0850001c00e89be`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:30.462878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:30.503662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.505102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.516885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1370/2724] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:21:30.539140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1355/2725] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:30.554854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:30.556803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:30.622275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406bde0-fbd6-4bb3-bd5e-8aecc701db8d service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:30.666773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:30.675531 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:30.676209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:30.676481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:30.676744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:30.688430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:30.689898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:30.700040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1371/2726] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:31.566604 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:31.580571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:31.622327 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c0545-be9d-4aa8-8473-282fe4498d65 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:31.626915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:31.627570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:31.627852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:31.628165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:31.630037 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:31.689569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:31.691017 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73da505-bb08-4d50-85eb-8b0aaa303f26 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:31.699776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1356/2727] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:38.543429 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:38.544825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:38.610885 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56005a8-b537-47f9-93ff-2b77e6424919 service glance] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:38.615992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:38.616602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:38.616900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:38.617169 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] PATH_INFO: `/v3/users/77226c1d77354e878897df51ef858df6` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:38.618518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77226c1d77354e878897df51ef858df6)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:38.620582 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:38.792816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77226c1d77354e878897df51ef858df6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:38.807417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77226c1d77354e878897df51ef858df6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:38.817074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77226c1d77354e878897df51ef858df6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:38.818034 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77226c1d77354e878897df51ef858df6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:38.818870 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77226c1d77354e878897df51ef858df6'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:38.897004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1372/2728] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:38 2022] DELETE /identity/v3/users/77226c1d77354e878897df51ef858df6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:38.901455 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:38.901925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:38.902051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:38.921548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:38.922786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:38.986947 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1357/2729] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:38.996613 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:38.997992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:39.057673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c0a5b0-ec0c-4154-b7c5-3f9fdf2844a7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:39.062088 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:39.062702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:39.062986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:39.063290 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:39.072653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:39.074058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:39.083329 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1373/2730] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:39.519521 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:39.520958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:39.533668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490e0f-f0eb-489b-a83f-348359223992 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:39.540921 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:39.541724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:39.542008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:39.542279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] PATH_INFO: `/v3/projects/fdd744502b6747ee83dd50bf6aa4a2fa` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:39.543432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd744502b6747ee83dd50bf6aa4a2fa)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:39.546026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:39.650554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd744502b6747ee83dd50bf6aa4a2fa', 'request_id': 'req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:39.657233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd744502b6747ee83dd50bf6aa4a2fa', 'request_id': 'req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:39.663511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd744502b6747ee83dd50bf6aa4a2fa', 'request_id': 'req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:39.665165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:39.755166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] The token cache is being invalidate because project fdd744502b6747ee83dd50bf6aa4a2fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:39.755373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:39.756092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:39.757824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1358/2731] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:39 2022] DELETE /identity/v3/projects/fdd744502b6747ee83dd50bf6aa4a2fa => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:39.762121 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:39.773171 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bbb1b-f9a1-4629-8896-5e7d09d6a626 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:39.777897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:39.829579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:39.830200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:39.830479 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:39.830751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] PATH_INFO: `/v3/users/f0ca8bdc5c214c4ea50e9d5db47243c1` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:39.831898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0ca8bdc5c214c4ea50e9d5db47243c1)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:39.833854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:40.003671 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0ca8bdc5c214c4ea50e9d5db47243c1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.016864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0ca8bdc5c214c4ea50e9d5db47243c1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.024259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0ca8bdc5c214c4ea50e9d5db47243c1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.024950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0ca8bdc5c214c4ea50e9d5db47243c1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.025548 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0ca8bdc5c214c4ea50e9d5db47243c1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.089261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1374/2732] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:39 2022] DELETE /identity/v3/users/f0ca8bdc5c214c4ea50e9d5db47243c1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:40.510061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:40.511540 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:40.525085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6eb58-e09f-4c29-814d-fc44d5cfbc57 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:40.584058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:40.584677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:40.584954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:40.585225 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] PATH_INFO: `/v3/projects/63debe76964d47bba6e4fa073f2a31ad` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:40.586368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63debe76964d47bba6e4fa073f2a31ad)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:40.589029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:40.697587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63debe76964d47bba6e4fa073f2a31ad', 'request_id': 'req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.703439 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63debe76964d47bba6e4fa073f2a31ad', 'request_id': 'req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.709756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63debe76964d47bba6e4fa073f2a31ad', 'request_id': 'req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.797302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] The token cache is being invalidate because project 63debe76964d47bba6e4fa073f2a31ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:40.797523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.798150 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:40.799666 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1359/2733] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:40 2022] DELETE /identity/v3/projects/63debe76964d47bba6e4fa073f2a31ad => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:40.803626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:40.805038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:40.817891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a2193-e2e7-4eab-8554-6aba04d2adf7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:40.874795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:40.875452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:40.875730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:40.876001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] PATH_INFO: `/v3/users/437921ba4bed4cd88cbb698ccf28cf3a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:40.877108 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=437921ba4bed4cd88cbb698ccf28cf3a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:40.879025 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:41.032180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:41.046185 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437921ba4bed4cd88cbb698ccf28cf3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.058486 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437921ba4bed4cd88cbb698ccf28cf3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.066287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437921ba4bed4cd88cbb698ccf28cf3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.066988 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437921ba4bed4cd88cbb698ccf28cf3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.067684 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-1f3cf857-2413-46ff-b1f2-fd736d101f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437921ba4bed4cd88cbb698ccf28cf3a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.153878 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1375/2734] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:40 2022] DELETE /identity/v3/users/437921ba4bed4cd88cbb698ccf28cf3a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:41.538809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:41.541388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:41.556761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7d72e-3bae-4e63-88b8-0b40c77e96c8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:41.616884 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:41.617440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:41.617723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:41.618005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] PATH_INFO: `/v3/projects/49208892ce7540a3b7d6b8a16474ed08` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:41.619153 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49208892ce7540a3b7d6b8a16474ed08)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:41.621085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:41.645502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:41.733879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49208892ce7540a3b7d6b8a16474ed08', 'request_id': 'req-4f10e1b8-693c-4d17-b81e-be86c74dde89'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.740331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49208892ce7540a3b7d6b8a16474ed08', 'request_id': 'req-4f10e1b8-693c-4d17-b81e-be86c74dde89'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.748322 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49208892ce7540a3b7d6b8a16474ed08', 'request_id': 'req-4f10e1b8-693c-4d17-b81e-be86c74dde89'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.810970 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] The token cache is being invalidate because project 49208892ce7540a3b7d6b8a16474ed08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:41.811297 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.811936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4f10e1b8-693c-4d17-b81e-be86c74dde89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:41.813472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1360/2735] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:41 2022] DELETE /identity/v3/projects/49208892ce7540a3b7d6b8a16474ed08 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:41.819442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:41.819933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:41.820036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:41.842474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:41.843751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:41.918933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1376/2736] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:41.928152 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:41.928434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:41.928707 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:41.948988 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:41.950191 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:41.959315 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1361/2737] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:41.964045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:41.974446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31708a-ab68-431c-9a72-9e981769ea5c None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:41.979052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:41.979738 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:41.980010 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:41.980280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:41.980916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:41.981325 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:41.982927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:41.999153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1377/2738] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:42.000564 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.010386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870b66e-0f3c-4d22-8443-857a6436ab8d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.016133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.016933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.017213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.017491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.018122 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.020078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.105609 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1362/2739] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:41 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:42.106670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.117445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b707952-467e-4d1a-8f43-4d4b850d848b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.178374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.178901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.179216 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.179491 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.180132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.182081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.270751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1378/2740] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:42 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:21:42.274824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.285509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e43c9-6313-4921-ba52-484f73cffd42 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.290778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.291400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.291677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.291945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.292575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.294167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.304306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1363/2741] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:42.308504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.319330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59df99-4284-4e25-be73-e8f7a1e74e71 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.323961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.324537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.324809 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.325076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.325705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.328131 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c38fa0-95d7-4cd6-8c7f-76afdc971536 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.339427 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1379/2742] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:42.344147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.354685 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a915c-1738-4a2a-bf42-67a0cb5a47f2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.359937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.360595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.360877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.361147 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] PATH_INFO: `/v3/projects/9f896a5b42d54a6c81859cbc1980a0ca/users/7bd8941f93434bcfb62d6695402343f1/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.376886 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f896a5b42d54a6c81859cbc1980a0ca, user_id=7bd8941f93434bcfb62d6695402343f1, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.379576 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.449723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1364/2743] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:42 2022] PUT /identity/v3/projects/9f896a5b42d54a6c81859cbc1980a0ca/users/7bd8941f93434bcfb62d6695402343f1/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:21:42.461910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.462301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.462626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.489376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:42.492161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] MFA Rules not processed for user `7bd8941f93434bcfb62d6695402343f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:42.493362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:42.559191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1380/2744] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:42.564871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.627419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a037c16-ae6d-4a76-a559-05f6f968b558 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.632065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.632658 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.632937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.633209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.633835 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.635806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.721824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1365/2745] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:42 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:42.725969 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.737536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a838-62c4-4abd-ade4-97d486eb3581 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.793262 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.793863 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.794142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.794415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.795602 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.797528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.894662 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1381/2746] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:42 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:21:42.898668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.909393 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326af81-2992-4ed0-87b5-674906c4d748 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.914071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.914661 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.914941 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.915249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.915887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.917488 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:42.929597 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1366/2747] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:42.933622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:42.944177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597298c-7043-479d-a929-a2fd1a21c2cc admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:42.948678 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:42.949310 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:42.949588 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:42.949859 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] PATH_INFO: `/v3/projects/4df6645486ab474a8efa044936e18b6a/users/c49e8c292cac41c5b299ad4af7a1e35b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:42.964554 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df6645486ab474a8efa044936e18b6a, user_id=c49e8c292cac41c5b299ad4af7a1e35b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:42.968222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:43.061226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1382/2748] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:42 2022] PUT /identity/v3/projects/4df6645486ab474a8efa044936e18b6a/users/c49e8c292cac41c5b299ad4af7a1e35b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:43.065369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:43.075334 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baea5f-0a08-4a70-a3ac-ae1f09ad6412 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:43.095204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:43.134893 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:43.135601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:43.135906 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:43.136180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:43.136858 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:43.139396 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8964a-de5f-4126-8482-39b4904b241c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:43.149763 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1367/2749] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:43.154284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:43.164875 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc1a49-be8e-4a48-be6b-e7beea4b08c2 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:43.171242 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:43.171967 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:43.172316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:43.172657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] PATH_INFO: `/v3/projects/4df6645486ab474a8efa044936e18b6a/users/c49e8c292cac41c5b299ad4af7a1e35b/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:43.174707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df6645486ab474a8efa044936e18b6a, user_id=c49e8c292cac41c5b299ad4af7a1e35b, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:43.177508 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:43.189375 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1383/2750] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:43 2022] PUT /identity/v3/projects/4df6645486ab474a8efa044936e18b6a/users/c49e8c292cac41c5b299ad4af7a1e35b/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:43.201675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:43.205649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:43.205649 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:43.233559 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] MFA Rules not processed for user `c49e8c292cac41c5b299ad4af7a1e35b`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:43.234710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:43.307049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1368/2751] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:43.318060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:43.319797 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:43.379314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e352dff-dfb8-4d23-835c-56db302d0a94 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:43.382990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:43.383627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:43.383908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:43.384180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:43.393585 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:43.395029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a3592-245f-46fd-a9db-6561e080338c service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:43.398937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:43.405634 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1384/2752] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:43.934058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:43.994821 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee9f98-0fca-4164-b1ce-cc734dbfcaee None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:43.999902 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:44.000513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:44.000790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:44.001057 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] PATH_INFO: `/v3/users/7bd8941f93434bcfb62d6695402343f1` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:44.002148 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd8941f93434bcfb62d6695402343f1)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:44.004187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:44.165299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:44.166882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd8941f93434bcfb62d6695402343f1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.179268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd8941f93434bcfb62d6695402343f1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.242935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd8941f93434bcfb62d6695402343f1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.243595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd8941f93434bcfb62d6695402343f1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.244238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd8941f93434bcfb62d6695402343f1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.258402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1369/2753] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:43 2022] DELETE /identity/v3/users/7bd8941f93434bcfb62d6695402343f1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:44.262822 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:44.263405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:44.263552 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:44.282516 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:44.283736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:44.343193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1385/2754] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:44.353003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:44.354422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:44.413476 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77559205-9976-49ee-ba30-d5a6499afd8c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:44.417176 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:44.417775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:44.418056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:44.418328 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:44.427753 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:44.429183 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:44.439160 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1370/2755] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:44.823939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:44.825500 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:44.840347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addba8e-0476-44f9-a860-63ade8ae42da None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:44.845873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:44.846503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:44.846891 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:44.847036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] PATH_INFO: `/v3/projects/9f896a5b42d54a6c81859cbc1980a0ca` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:44.848213 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f896a5b42d54a6c81859cbc1980a0ca)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:44.850372 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:44.942635 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f896a5b42d54a6c81859cbc1980a0ca', 'request_id': 'req-506f7e97-f355-4d10-92e0-79a57d319353'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.988038 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f896a5b42d54a6c81859cbc1980a0ca', 'request_id': 'req-506f7e97-f355-4d10-92e0-79a57d319353'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:44.994541 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f896a5b42d54a6c81859cbc1980a0ca', 'request_id': 'req-506f7e97-f355-4d10-92e0-79a57d319353'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.005861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] The token cache is being invalidate because project 9f896a5b42d54a6c81859cbc1980a0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:45.006148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.006780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.008289 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1386/2756] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:44 2022] DELETE /identity/v3/projects/9f896a5b42d54a6c81859cbc1980a0ca => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:45.012253 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:45.022307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc45b9-908f-47c4-85ce-5640acf9cb6f service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:45.078504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:45.079063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:45.079377 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:45.079645 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] PATH_INFO: `/v3/users/c49e8c292cac41c5b299ad4af7a1e35b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:45.080758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49e8c292cac41c5b299ad4af7a1e35b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:45.082664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:45.259464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49e8c292cac41c5b299ad4af7a1e35b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.272070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49e8c292cac41c5b299ad4af7a1e35b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.274933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:45.358034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49e8c292cac41c5b299ad4af7a1e35b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.358599 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49e8c292cac41c5b299ad4af7a1e35b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.359246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-416be38c-c0b8-4ede-a211-58228c8b4656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49e8c292cac41c5b299ad4af7a1e35b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:45.372657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1371/2757] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:21:45 2022] DELETE /identity/v3/users/c49e8c292cac41c5b299ad4af7a1e35b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:45.943920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:45.952752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:45.960517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7e97-f355-4d10-92e0-79a57d319353 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.019788 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.020443 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.020721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.021090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] PATH_INFO: `/v3/projects/4df6645486ab474a8efa044936e18b6a` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.022157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df6645486ab474a8efa044936e18b6a)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.024184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.050232 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:46.125961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df6645486ab474a8efa044936e18b6a', 'request_id': 'req-b8c225b2-1452-4a60-b1c3-031cb5548510'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:46.188844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df6645486ab474a8efa044936e18b6a', 'request_id': 'req-b8c225b2-1452-4a60-b1c3-031cb5548510'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:46.195975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df6645486ab474a8efa044936e18b6a', 'request_id': 'req-b8c225b2-1452-4a60-b1c3-031cb5548510'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:46.208180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] The token cache is being invalidate because project 4df6645486ab474a8efa044936e18b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:21:46.208471 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:46.209125 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b8c225b2-1452-4a60-b1c3-031cb5548510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:21:46.210653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1387/2758] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:21:45 2022] DELETE /identity/v3/projects/4df6645486ab474a8efa044936e18b6a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:46.215861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.216454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.216454 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.237749 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:46.239156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:46.302179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1372/2759] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:46.310087 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.310430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.310713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.332625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:46.333950 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:46.342138 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1388/2760] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:46.347368 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.357347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261ed85-b1be-4285-beee-334d044f3b4f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.363240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.363849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.364124 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.364422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.365087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:21:46.365390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.367080 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.379363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1373/2761] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:21:46.383551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.395347 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de6cdb-78e1-442f-bab3-235ff77e8ba3 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.402291 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.402892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.403184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.403453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.404077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.406568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.490434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1389/2762] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:21:46 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:46.494816 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.505653 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac01cb8-1131-4e76-8bbb-bcc381387b51 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.565072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.565680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.565962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.566347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.566983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.568937 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.668472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1374/2763] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:21:46 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:21:46.672872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.683511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a925854-34c9-4655-9615-9cc553ad44d7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.688772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.689364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.689641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.689911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.690550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.692182 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.703708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1390/2764] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:21:46.707914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.718750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56565bb9-e41a-4921-9ddc-71cf6977bac9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.723575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.724169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.724448 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.724722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.725390 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.727007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b111184-2f16-4d96-8a65-8a605b1373e0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.737968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1375/2765] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:21:46.748597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:46.752838 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba375a93-2f5b-424d-9e3b-96fd5c056b76 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:46.757547 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:46.758179 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.758452 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.758720 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] PATH_INFO: `/v3/projects/687ce7c5fe4e4609a7eeb3839270d507/users/449b28d341aa43f5a01dcce970bfa951/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.774767 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=687ce7c5fe4e4609a7eeb3839270d507, user_id=449b28d341aa43f5a01dcce970bfa951, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:46.777955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:46.867816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1391/2766] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:21:46 2022] PUT /identity/v3/projects/687ce7c5fe4e4609a7eeb3839270d507/users/449b28d341aa43f5a01dcce970bfa951/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:21:46.935826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:46.936106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:46.936374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:46.962913 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] MFA Rules not processed for user `449b28d341aa43f5a01dcce970bfa951`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:21:46.964385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:47.008587 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:47.032433 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1376/2767] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:21:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:21:47.043407 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:21:47.045142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:21:47.123009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dfab4-0d2b-477c-b91a-97e40a0f0ade admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:21:47.127236 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1c10f44607a4d8080891c89b9b88345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:21:47.127964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:21:47.128267 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:21:47.128569 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:21:47.138292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:21:47.140434 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d938a6b-3452-4d53-bc0c-c05fc53a72b9 service glance] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:21:47.149449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1392/2768] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:21:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:21:57.145263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=77,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:21:57.176348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=9 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:14.296975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:14.298914 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:14.363923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c67152-7115-4ef1-a6fd-a9e69bd41d0d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:14.368314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:14.368915 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:14.369194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:14.369485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] PATH_INFO: `/v3/users/831c626055c94fa786ea1eee250025b9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:14.370579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831c626055c94fa786ea1eee250025b9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:14.372504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:14.528381 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831c626055c94fa786ea1eee250025b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:14.540752 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831c626055c94fa786ea1eee250025b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:14.612943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831c626055c94fa786ea1eee250025b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:14.613538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831c626055c94fa786ea1eee250025b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:14.614275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831c626055c94fa786ea1eee250025b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:14.626824 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1377/2769] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:14 2022] DELETE /identity/v3/users/831c626055c94fa786ea1eee250025b9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:14.631281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:14.631741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:14.632530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:14.651253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:14.652456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:14.710411 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1393/2770] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:14.721106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:14.722510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:14.778588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a443887-2b5c-4f7e-85fd-31d1f63395d8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:14.782198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:14.783464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:14.783742 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:14.784012 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:14.792827 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:14.794240 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:14.803535 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1378/2771] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:15.209000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:15.212862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:15.229861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba6564-336a-490c-b200-b50f8ead91d4 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:15.241175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:15.246580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:15.246880 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:15.247170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] PATH_INFO: `/v3/projects/2db2674ebe3d4eb98c5653f08d0ac0ea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:15.248608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db2674ebe3d4eb98c5653f08d0ac0ea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:15.250521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:15.376525 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db2674ebe3d4eb98c5653f08d0ac0ea', 'request_id': 'req-29609785-cb3f-4149-8089-7121fdbe0836'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.466526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db2674ebe3d4eb98c5653f08d0ac0ea', 'request_id': 'req-29609785-cb3f-4149-8089-7121fdbe0836'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.473990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db2674ebe3d4eb98c5653f08d0ac0ea', 'request_id': 'req-29609785-cb3f-4149-8089-7121fdbe0836'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.484252 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] The token cache is being invalidate because project 2db2674ebe3d4eb98c5653f08d0ac0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:15.484536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.485163 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.486696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1394/2772] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:15 2022] DELETE /identity/v3/projects/2db2674ebe3d4eb98c5653f08d0ac0ea => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:15.490688 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:15.500948 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af594d7c-7180-41ec-8974-9fa64d45e3f0 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:15.519790 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:15.558703 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:15.559379 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:15.559668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:15.559942 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] PATH_INFO: `/v3/users/59680166042d47ef8abeac0e5c036fce` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:15.561092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59680166042d47ef8abeac0e5c036fce)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:15.563054 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:15.829156 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59680166042d47ef8abeac0e5c036fce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.848584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59680166042d47ef8abeac0e5c036fce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.934895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59680166042d47ef8abeac0e5c036fce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.935554 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59680166042d47ef8abeac0e5c036fce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.936170 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1c89eecc-14a2-49c5-9a4c-2b226a3e67c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59680166042d47ef8abeac0e5c036fce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:15.949936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1379/2773] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:15 2022] DELETE /identity/v3/users/59680166042d47ef8abeac0e5c036fce => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:16.462240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:16.463714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:16.477268 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29609785-cb3f-4149-8089-7121fdbe0836 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:16.534050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:16.534673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:16.534949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:16.535255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] PATH_INFO: `/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:16.536345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e5337ec4b448fdb79b47cce3bcc7f0)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:16.538226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:16.621760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:16.637317 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e5337ec4b448fdb79b47cce3bcc7f0', 'request_id': 'req-a9965d54-f73e-4ae5-86d4-7b2bff320471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:16.717279 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e5337ec4b448fdb79b47cce3bcc7f0', 'request_id': 'req-a9965d54-f73e-4ae5-86d4-7b2bff320471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:16.724540 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e5337ec4b448fdb79b47cce3bcc7f0', 'request_id': 'req-a9965d54-f73e-4ae5-86d4-7b2bff320471'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:16.735318 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] The token cache is being invalidate because project f9e5337ec4b448fdb79b47cce3bcc7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:16.736713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:16.737210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-a9965d54-f73e-4ae5-86d4-7b2bff320471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:16.738699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1395/2774] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:16 2022] DELETE /identity/v3/projects/f9e5337ec4b448fdb79b47cce3bcc7f0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:16.743795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:16.744289 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:16.744400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:16.763936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:16.767892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:16.825644 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1380/2775] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:16.834235 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:16.836626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:16.836626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:16.855648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:16.859561 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:16.865964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1396/2776] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:16.870260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:16.880386 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450ae8c-d7b7-4c3e-a39f-a7cdce8d1f94 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:16.884945 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:16.894072 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:16.900272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1381/2777] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:22:16.904475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:16.914908 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6a998-d7ed-41e2-b12d-cc5e14003d22 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:16.919590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:16.920196 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:16.920478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:16.920750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:16.921423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:16.924197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.027866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1397/2778] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:16 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:17.031916 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.042399 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7812a2-5b45-4db1-9e47-a6d330e8e420 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.067365 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:17.097677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.098261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.098539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.098811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.099509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.101993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.177891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1382/2779] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:17 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:17.182204 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.192773 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d880c-5ab8-4ee0-996c-6e4491c6b20f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.197376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.197961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.198240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.198509 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.199175 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.200789 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.211872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1398/2780] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:17.215895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.226880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c338b0d-8841-4658-9845-59d870663d7b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.231213 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.231788 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.232137 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.232417 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.233040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.234663 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00175af-700f-40f3-9b36-196597507ff1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.244808 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1383/2781] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:17.251639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.259080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c203f-433a-4247-a4fa-921d93810df7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.263801 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.264430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.264709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.265029 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] PATH_INFO: `/v3/projects/5c13ad8d7c1a480e960094b5017725c0/users/c0de61032036417abfe13d35ec1fc47d/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.283968 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c13ad8d7c1a480e960094b5017725c0, user_id=c0de61032036417abfe13d35ec1fc47d, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.284927 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.375000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1399/2782] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:17 2022] PUT /identity/v3/projects/5c13ad8d7c1a480e960094b5017725c0/users/c0de61032036417abfe13d35ec1fc47d/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:17.386237 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.386533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.386826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.418470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] MFA Rules not processed for user `c0de61032036417abfe13d35ec1fc47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:17.419811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:17.484817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1384/2783] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:17.489390 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.552610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494cdb-9ce9-493f-946b-7cf411d16b8b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.557259 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.557866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.558151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.558422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.559059 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.563020 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.638900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1400/2784] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:17 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:17.643608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.654241 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af86ca3-f887-467b-8536-0ddde39269b0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.708607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.709193 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.709470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.709740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.710347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.712544 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.791308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1385/2785] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:17 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:22:17.800418 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.811007 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87dfea-ebbc-4f6f-99ee-8c2685a7cd47 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.816112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.816701 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.816978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.817250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.817886 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.819535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.831879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1401/2786] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:17.835925 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.846932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6a6b8-0176-4f54-a637-ffcc6a1427b0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.851622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:17.852259 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:17.852541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:17.852815 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] PATH_INFO: `/v3/projects/7508361e232949f0beb48a624b378b2f/users/4422079ebf724785b647735aaa2ddb65/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:17.867432 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7508361e232949f0beb48a624b378b2f, user_id=4422079ebf724785b647735aaa2ddb65, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:17.870098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:17.937511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1386/2787] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:17 2022] PUT /identity/v3/projects/7508361e232949f0beb48a624b378b2f/users/4422079ebf724785b647735aaa2ddb65/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:22:17.941719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:17.951600 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc7ea4-b357-46f8-9518-013558b69eee admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:17.973713 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:18.011595 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:18.012097 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:18.012374 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:18.012647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:18.013276 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:18.014852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326659a-ec3a-4f20-b3e4-73f6b80438d0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:18.027644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1402/2788] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:18.031853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:18.038028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:18.043351 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bfbd2-5970-4709-83a0-d39bfaffc19f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:18.048757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:18.049410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:18.049854 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:18.050134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] PATH_INFO: `/v3/projects/7508361e232949f0beb48a624b378b2f/users/4422079ebf724785b647735aaa2ddb65/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:18.052123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7508361e232949f0beb48a624b378b2f, user_id=4422079ebf724785b647735aaa2ddb65, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:18.054736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:18.065402 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1387/2789] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:18 2022] PUT /identity/v3/projects/7508361e232949f0beb48a624b378b2f/users/4422079ebf724785b647735aaa2ddb65/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:22:18.075481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:18.076530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:18.076530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:18.104818 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] MFA Rules not processed for user `4422079ebf724785b647735aaa2ddb65`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:18.106066 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:18.174874 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1403/2790] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:18.186463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:18.188167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:18.248400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf9bc5-4806-49a8-8279-4d035445cdfb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:18.252184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:18.252782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:18.253063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:18.253337 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:18.262145 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:18.263636 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:18.273027 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1388/2791] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:18.423449 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:18.433481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662144ec-941a-48a0-b9a5-ac9179cfd82d None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:18.445478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:18.445478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:18.445478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:18.445478 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:18.518872 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:18.520364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f7da5-7d9e-4d33-b47e-4075d3af13ac service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:18.529535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1404/2792] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:19.043857 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:19.105530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245a9a2-6b88-4fd5-a86a-596147740b53 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:19.110104 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:19.110891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:19.111194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:19.111470 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] PATH_INFO: `/v3/users/c0de61032036417abfe13d35ec1fc47d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:19.112573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0de61032036417abfe13d35ec1fc47d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:19.114510 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:19.317954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0de61032036417abfe13d35ec1fc47d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:19.330111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0de61032036417abfe13d35ec1fc47d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:19.400461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0de61032036417abfe13d35ec1fc47d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:19.401084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0de61032036417abfe13d35ec1fc47d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:19.401694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0de61032036417abfe13d35ec1fc47d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:19.414330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1389/2793] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:19 2022] DELETE /identity/v3/users/c0de61032036417abfe13d35ec1fc47d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:19.418825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:19.419223 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:19.423978 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:19.438807 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:19.440014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:19.480079 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=100 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:19.498841 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1405/2794] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:19.509642 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:19.511079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:19.527509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:19.580025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca403-8120-4f9b-8117-a7861967b2b7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:19.580025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:19.580410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:19.580410 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:19.580665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:19.589847 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:19.591306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:19.601704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1390/2795] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:20.033536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:20.035226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:20.050074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f86c5-8a29-47ea-9124-ce2d0863c2ae None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:20.055659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:20.056344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:20.057152 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:20.057435 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] PATH_INFO: `/v3/projects/5c13ad8d7c1a480e960094b5017725c0` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:20.058813 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c13ad8d7c1a480e960094b5017725c0)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:20.061323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:20.162744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c13ad8d7c1a480e960094b5017725c0', 'request_id': 'req-743dc186-20f4-4caa-af8d-ada4643799ac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.232099 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c13ad8d7c1a480e960094b5017725c0', 'request_id': 'req-743dc186-20f4-4caa-af8d-ada4643799ac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.239592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c13ad8d7c1a480e960094b5017725c0', 'request_id': 'req-743dc186-20f4-4caa-af8d-ada4643799ac'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.249711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] The token cache is being invalidate because project 5c13ad8d7c1a480e960094b5017725c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:20.249993 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.250619 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.252722 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1406/2796] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:20 2022] DELETE /identity/v3/projects/5c13ad8d7c1a480e960094b5017725c0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:20.256681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:20.266569 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14260-c528-4d70-9645-2cb72e43b0f8 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:20.327016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:20.327693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:20.327968 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:20.328264 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] PATH_INFO: `/v3/users/4422079ebf724785b647735aaa2ddb65` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:20.329490 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4422079ebf724785b647735aaa2ddb65)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:20.331507 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:20.462404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4422079ebf724785b647735aaa2ddb65'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.474789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4422079ebf724785b647735aaa2ddb65'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.540279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4422079ebf724785b647735aaa2ddb65'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.540897 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4422079ebf724785b647735aaa2ddb65'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.541588 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4422079ebf724785b647735aaa2ddb65'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:20.558933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1391/2797] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:20 2022] DELETE /identity/v3/users/4422079ebf724785b647735aaa2ddb65 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:20.950474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:20.951960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:20.964948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dc186-20f4-4caa-af8d-ada4643799ac admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:21.021448 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:21.022071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:21.022344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:21.022745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] PATH_INFO: `/v3/projects/7508361e232949f0beb48a624b378b2f` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:21.023873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7508361e232949f0beb48a624b378b2f)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:21.025768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:21.163474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7508361e232949f0beb48a624b378b2f', 'request_id': 'req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:21.214201 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7508361e232949f0beb48a624b378b2f', 'request_id': 'req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:21.221080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7508361e232949f0beb48a624b378b2f', 'request_id': 'req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:21.227495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:21.231832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] The token cache is being invalidate because project 7508361e232949f0beb48a624b378b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:21.232118 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:21.232742 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-6aaa1998-8c50-48f9-b3f2-32c86adc4a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:21.234220 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1407/2798] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:20 2022] DELETE /identity/v3/projects/7508361e232949f0beb48a624b378b2f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:23.369111 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:23.370562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:23.383633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775daa63-d36e-4190-94d4-67ac76a7d0d2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:23.412962 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:23.438894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:23.439733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:23.440224 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:23.440927 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] PATH_INFO: `/v3/users/449b28d341aa43f5a01dcce970bfa951` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:23.442063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=449b28d341aa43f5a01dcce970bfa951)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:23.444081 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:23.614823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449b28d341aa43f5a01dcce970bfa951'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:23.627088 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449b28d341aa43f5a01dcce970bfa951'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:23.700051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449b28d341aa43f5a01dcce970bfa951'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:23.700671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449b28d341aa43f5a01dcce970bfa951'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:23.701293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449b28d341aa43f5a01dcce970bfa951'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:23.714744 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1392/2799] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:23 2022] DELETE /identity/v3/users/449b28d341aa43f5a01dcce970bfa951 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:23.719263 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:23.719747 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:23.720191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:23.739604 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:23.749915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:23.800037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1408/2800] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:23.809553 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:23.810977 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:23.868048 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec57ba8-9553-445b-9fa9-cdabaaad4457 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:23.871634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:23.872232 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:23.872508 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:23.872775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:23.882179 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:23.883634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddaf03-f02a-4bfa-acf3-845d0c8fa707 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:23.892901 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1393/2801] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:24.310518 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:24.311883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:24.323970 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350f0f9-5066-4d62-bb64-c8c7c4580748 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:24.329197 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:24.330098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.330550 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.330898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] PATH_INFO: `/v3/projects/687ce7c5fe4e4609a7eeb3839270d507` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.332971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687ce7c5fe4e4609a7eeb3839270d507)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:24.334938 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:24.477965 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687ce7c5fe4e4609a7eeb3839270d507', 'request_id': 'req-88965659-b683-48fd-920d-ea512f0b73f1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:24.534494 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687ce7c5fe4e4609a7eeb3839270d507', 'request_id': 'req-88965659-b683-48fd-920d-ea512f0b73f1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:24.541592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687ce7c5fe4e4609a7eeb3839270d507', 'request_id': 'req-88965659-b683-48fd-920d-ea512f0b73f1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:24.553781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] The token cache is being invalidate because project 687ce7c5fe4e4609a7eeb3839270d507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:24.554067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:24.554784 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-88965659-b683-48fd-920d-ea512f0b73f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:24.556398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1409/2802] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:24 2022] DELETE /identity/v3/projects/687ce7c5fe4e4609a7eeb3839270d507 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:24.565622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.565622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.565622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.584908 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:24.586361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:24.633581 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:24.645093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1394/2803] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:22:24.652862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.653148 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.653417 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.673271 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:24.674446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:24.682278 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1410/2804] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:22:24.686489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:24.696933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092ac7d-5458-47e5-b7fd-a1a765eefb2c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:24.701511 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:24.702094 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.702370 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.702639 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.703306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:22:24.703626 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:24.705347 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:24.717233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1395/2805] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:22:24.721542 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:24.730850 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5787ea-be57-4651-a639-28eb76fd83ea None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:24.736133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:24.736814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.737104 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.737370 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.738021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:24.739964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:24.822339 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1411/2806] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:24 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:24.826369 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:24.836954 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4166c-d373-4f1c-8953-dbc643d3702e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:24.892531 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:24.893155 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.893429 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.893694 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.894307 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:24.896258 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:24.968058 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1396/2807] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:24 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:24.972683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:24.983647 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea0b4-50dc-446b-9b0b-3e0e1970bdf6 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:24.989156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:24.989905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:24.990178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:24.990442 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:24.991078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:24.992686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:25.003192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1412/2808] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:25.007215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:25.017440 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bac202-b141-46e8-b272-371cda4d3614 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:25.022395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:25.022990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:25.023303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:25.023580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:25.024245 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:25.025987 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c70e-88bd-4fc5-8d64-845554a65eed admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:25.040939 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1397/2809] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:25.042090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:25.052723 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce615e8-fcad-46b2-a7c2-28a6ed2e90b7 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:25.057704 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:25.058332 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:25.058611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:25.058884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] PATH_INFO: `/v3/projects/b5b81c24bc4944e790642578b7fa1131/users/38fa019186ce4539bbcc03d1d76f6e2b/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:25.077599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b81c24bc4944e790642578b7fa1131, user_id=38fa019186ce4539bbcc03d1d76f6e2b, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:25.077599 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:25.140353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1413/2810] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:25 2022] PUT /identity/v3/projects/b5b81c24bc4944e790642578b7fa1131/users/38fa019186ce4539bbcc03d1d76f6e2b/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:25.152013 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:25.152303 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:25.152579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:25.181261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] MFA Rules not processed for user `38fa019186ce4539bbcc03d1d76f6e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:25.182477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:25.252374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1398/2811] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:25.268881 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:25.270526 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:25.291076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:25.327530 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a0e7-e025-4fa8-a62f-efb268950ffe admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:25.333320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:25.333320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:25.333320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:25.333320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:25.341284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:25.342721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004febc2-f020-4e34-8e73-730f9a82392c service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:25.352447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1414/2812] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:28.766485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:28.768861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:28.820000 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:28.834478 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0a5c0-bc89-4276-95f0-72849eab6fed None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:28.839388 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:28.843712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:28.843712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:28.843712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] PATH_INFO: `/v3/users/55c65749779c48e8b0850001c00e89be` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:28.843712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c65749779c48e8b0850001c00e89be)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:28.844299 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:29.059735 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c65749779c48e8b0850001c00e89be'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.072065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c65749779c48e8b0850001c00e89be'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.148891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c65749779c48e8b0850001c00e89be'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.149528 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c65749779c48e8b0850001c00e89be'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.150194 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c65749779c48e8b0850001c00e89be'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.162731 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1399/2813] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:28 2022] DELETE /identity/v3/users/55c65749779c48e8b0850001c00e89be => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:29.167155 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:29.167637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:29.167746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:29.187570 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:29.188879 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:29.249323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1415/2814] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:29.259743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:29.261205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:29.319235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345dff3-fa09-49a7-8c1f-7d6ccfb67eb9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:29.322777 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:29.323419 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:29.323701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:29.323974 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:29.333065 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:29.334482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cfa42-73f6-4dee-ab36-98435b9610a1 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:29.344202 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1400/2815] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:29.744412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:29.746422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:29.758362 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df459b7-486c-4d30-b51a-13ebcbf8eaf2 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:29.763222 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:29.763856 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:29.764135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:29.764960 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] PATH_INFO: `/v3/projects/9d7ff71e23bb4a75b2c84b2a6c716bbe` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:29.765592 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7ff71e23bb4a75b2c84b2a6c716bbe)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:29.767601 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:29.874456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7ff71e23bb4a75b2c84b2a6c716bbe', 'request_id': 'req-f3ca3381-b936-4a21-9a55-ab377cead69a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.925719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7ff71e23bb4a75b2c84b2a6c716bbe', 'request_id': 'req-f3ca3381-b936-4a21-9a55-ab377cead69a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.932198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7ff71e23bb4a75b2c84b2a6c716bbe', 'request_id': 'req-f3ca3381-b936-4a21-9a55-ab377cead69a'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.942608 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] The token cache is being invalidate because project 9d7ff71e23bb4a75b2c84b2a6c716bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:29.942922 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.943756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f3ca3381-b936-4a21-9a55-ab377cead69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:29.945249 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1416/2816] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:29 2022] DELETE /identity/v3/projects/9d7ff71e23bb4a75b2c84b2a6c716bbe => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:29.950251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:29.950712 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:29.950826 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:29.970882 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:29.972983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:30.030228 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1401/2817] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:30.038128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.038436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.038705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.058392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:30.059625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:30.068166 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1417/2818] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:30.072416 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.082628 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89876-747c-4c5d-9800-7475f427c667 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.087471 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.088056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.088336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.088611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.089252 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:22:30.089570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.091181 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.099693 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:30.102952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1402/2819] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:22:30.107074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.116341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349a050-a327-407c-bd7a-b0f78b50d689 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.121016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.121593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.121866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.122129 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.122749 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.124711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.227642 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1418/2820] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:30 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:30.231195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.243414 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251d4d6-f2c8-462d-af37-04be109fc94d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.302246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.302883 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.303187 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.303460 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.304092 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.306004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.414407 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1403/2821] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:30 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:30.418539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.429661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad875a-4d7c-4c7a-8b75-f381bc404bb1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.434371 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.434958 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.435281 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.435620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.436261 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.441218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.448746 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1419/2822] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:30.452971 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.463372 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0ccb9-d219-45ed-be12-0d20a2458a45 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.468212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.468796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.469852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.469852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.470071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.471705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55588ca-44d0-4ff4-8659-acbea162f9b6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.482495 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1404/2823] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:30.486772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.496708 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:30.497640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb80d-8bba-4864-9e22-462e6319a920 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.502719 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.503732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.504071 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.504340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] PATH_INFO: `/v3/projects/8c703a441ccd4fd6aed56096783f1086/users/7538ff30b3ff42a784da0336012f901d/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.519297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c703a441ccd4fd6aed56096783f1086, user_id=7538ff30b3ff42a784da0336012f901d, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.522022 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.629861 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1420/2824] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:30 2022] PUT /identity/v3/projects/8c703a441ccd4fd6aed56096783f1086/users/7538ff30b3ff42a784da0336012f901d/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:22:30.635527 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb86ff9-465b-4980-b32d-7fb720c710ea None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.635832 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb86ff9-465b-4980-b32d-7fb720c710ea None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.639946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb86ff9-465b-4980-b32d-7fb720c710ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.655157 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-feb86ff9-465b-4980-b32d-7fb720c710ea None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:30.656451 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-feb86ff9-465b-4980-b32d-7fb720c710ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:30.717400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1405/2825] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:30.727598 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:30.729287 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:30.791178 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3caf3d-6845-471e-a69a-7354da2e2109 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:30.794916 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:30.795559 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:30.795836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:30.796102 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:30.815673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:30.815673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:30.818795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1421/2826] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:32.105723 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f62e5-b6a6-4ce6-a1bc-8df7e979b832 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:32.106208 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f62e5-b6a6-4ce6-a1bc-8df7e979b832 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:32.106608 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f62e5-b6a6-4ce6-a1bc-8df7e979b832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:32.140647 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-940f62e5-b6a6-4ce6-a1bc-8df7e979b832 None None] MFA Rules not processed for user `7538ff30b3ff42a784da0336012f901d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:32.141992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-940f62e5-b6a6-4ce6-a1bc-8df7e979b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:32.217634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1406/2827] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:32.228781 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:32.230224 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:32.293535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b7d7f-e818-4967-a464-dbf48b542f58 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:32.297611 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:32.298226 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:32.298505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:32.298777 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:32.308351 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:32.309828 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:32.321892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1422/2828] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:34.262268 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:34.262823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:34.262823 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:34.283350 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] MFA Rules not processed for user `7538ff30b3ff42a784da0336012f901d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:34.285140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:34.292098 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1407/2829] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:34.921721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:34.923183 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:34.996912 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfb7c2-3f32-4159-b0ae-5378f48b58b9 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:35.002033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:35.003300 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:35.003580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:35.003849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:35.005563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:35.016928 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:35.019424 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a17c-8038-4b9a-b082-6b68c7d9d26d service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:35.029702 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1423/2830] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:36.752169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:36.754338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:36.762338 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:36.767899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ecce-80dd-440b-bcfb-44e08ab22a54 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:36.772993 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:36.773611 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:36.773888 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:36.774154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] PATH_INFO: `/v3/users/38fa019186ce4539bbcc03d1d76f6e2b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:36.775284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fa019186ce4539bbcc03d1d76f6e2b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:36.777184 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:36.991060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fa019186ce4539bbcc03d1d76f6e2b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:37.003804 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fa019186ce4539bbcc03d1d76f6e2b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:37.080020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fa019186ce4539bbcc03d1d76f6e2b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:37.080686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fa019186ce4539bbcc03d1d76f6e2b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:37.081795 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fa019186ce4539bbcc03d1d76f6e2b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:37.095548 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1408/2831] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:36 2022] DELETE /identity/v3/users/38fa019186ce4539bbcc03d1d76f6e2b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:37.100445 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:37.100939 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:37.101069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:37.121718 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:37.122982 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:37.135164 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:37.195135 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1424/2832] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:22:37.206570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:37.208083 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:37.274206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1cfa8-d7b0-4293-ace7-f568c9d1a71d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:37.277946 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:37.278556 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:37.278841 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:37.279133 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:37.291761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:37.293209 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056c0d-30cf-43ef-b9c3-b178c886e1e6 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:37.304373 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1409/2833] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:22:37.796199 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:37.798067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:37.812014 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298d7a-0135-4bcd-bbae-7a894b40117e None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:37.818363 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:37.819048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:37.819934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:37.820210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] PATH_INFO: `/v3/projects/b5b81c24bc4944e790642578b7fa1131` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:37.821640 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b81c24bc4944e790642578b7fa1131)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:37.823544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:37.953002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b81c24bc4944e790642578b7fa1131', 'request_id': 'req-f6866ecd-1054-4726-b6a1-e2a90dec6af1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:38.019573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b81c24bc4944e790642578b7fa1131', 'request_id': 'req-f6866ecd-1054-4726-b6a1-e2a90dec6af1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:38.026439 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b81c24bc4944e790642578b7fa1131', 'request_id': 'req-f6866ecd-1054-4726-b6a1-e2a90dec6af1'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:38.036725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] The token cache is being invalidate because project b5b81c24bc4944e790642578b7fa1131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:38.037011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:38.037633 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:38.039128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1425/2834] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:37 2022] DELETE /identity/v3/projects/b5b81c24bc4944e790642578b7fa1131 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:38.044293 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6538-903b-44eb-bcd9-b8851d1539e1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.044755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6538-903b-44eb-bcd9-b8851d1539e1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.044881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6538-903b-44eb-bcd9-b8851d1539e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.065607 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-049b6538-903b-44eb-bcd9-b8851d1539e1 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:38.066768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-049b6538-903b-44eb-bcd9-b8851d1539e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.110192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.114198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.126570 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1410/2835] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:38.135206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.136928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.136928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.155026 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:38.156266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.164415 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1411/2836] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:22:38.168555 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.170462 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6866ecd-1054-4726-b6a1-e2a90dec6af1 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.175295 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.175917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.176198 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.176474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.178227 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.178348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669ed83-4655-4bf5-97dc-2fc2024360c8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.182976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.186625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.186625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.186625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.186625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:22:38.186625 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.187084 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.188012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.189441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.194404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:38.198519 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1426/2837] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:38.200362 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1412/2838] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:22:38.204626 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.216218 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84676d-f02c-4bfa-8ea4-1c88d2c0431b service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.220778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.221543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.221905 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.222181 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.222844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.226353 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.342563 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1427/2839] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:38 2022] POST /identity/v3/projects => generated 399 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:38.347472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.357773 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1122c7-f99f-49ab-ad5f-711ff469f765 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.424049 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.425397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.425676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.425952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.426674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.428643 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.538562 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1413/2840] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:38 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:38.542522 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.554398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6994d38-f92d-4b69-93ab-5a537794c657 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.559240 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.559971 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.560194 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.560470 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.561139 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.562745 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.574824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1428/2841] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:38.578992 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.589701 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da19c0-148c-41f7-9a1c-ccc94985f925 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.595068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.595711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.595990 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.596864 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.597669 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.599287 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708bc4d-2582-4b14-9554-82b30afb5f0d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.609961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1414/2842] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:38.614575 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.624669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77695-08e4-47b5-8ccd-b23771ca65e2 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.629705 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.630457 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.630810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.631180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] PATH_INFO: `/v3/projects/d4d061b959df40babbff24d17d151b19/users/827e05559bd4410484ca4489e1ef967d/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.634579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:38.646919 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d061b959df40babbff24d17d151b19, user_id=827e05559bd4410484ca4489e1ef967d, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.649802 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.717505 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1429/2843] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:38 2022] PUT /identity/v3/projects/d4d061b959df40babbff24d17d151b19/users/827e05559bd4410484ca4489e1ef967d/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:38.728783 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.729089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.729363 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.759873 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] MFA Rules not processed for user `827e05559bd4410484ca4489e1ef967d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:38.761063 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.830025 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1415/2844] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:38.842469 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:38.844606 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:38.905627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d93b0d-24a5-4d9c-8417-a7637f5883e0 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:38.911123 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:38.911913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:38.912286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:38.912656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:38.923866 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:38.925485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:38.937333 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1430/2845] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:47.914040 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:47.916014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:47.996196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:48.001024 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6225a1-4cbe-45f7-b7e4-30f13d95156d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:48.007117 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:48.012638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:48.012638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:48.012638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:48.012638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:48.022716 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:48.024489 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba96b5-72df-4a13-8a4e-99587c446b30 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:48.036513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1416/2846] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:48.932973 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:49.017963 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:49.083804 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eb86d-3322-41f3-8b14-8dcd54cf4acf service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:49.088692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:49.089321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:49.089597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:49.089868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] PATH_INFO: `/v3/users/827e05559bd4410484ca4489e1ef967d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:49.090990 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=827e05559bd4410484ca4489e1ef967d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:49.093037 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:49.246934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827e05559bd4410484ca4489e1ef967d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:49.259898 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827e05559bd4410484ca4489e1ef967d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:49.266726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827e05559bd4410484ca4489e1ef967d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:49.267527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827e05559bd4410484ca4489e1ef967d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:49.268161 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827e05559bd4410484ca4489e1ef967d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:49.336288 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1431/2847] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:22:49 2022] DELETE /identity/v3/users/827e05559bd4410484ca4489e1ef967d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:49.340871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:49.341355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:49.341464 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:49.360871 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:49.362068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:49.420840 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1417/2848] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:49.430693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:49.432170 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:49.489487 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d5786-2780-4561-8f3d-d85e05285763 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:49.493127 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:49.493724 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:49.494001 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:49.494270 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:49.503117 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:49.505009 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769efd7-f09d-448d-8b8b-2cc0c20bacda service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:49.514292 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1432/2849] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:49.983621 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:49.985069 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:49.998573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acf211-e0a5-423a-b1a3-691f108d516c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.005943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.005943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.005943 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.006348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] PATH_INFO: `/v3/projects/d4d061b959df40babbff24d17d151b19` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.007662 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d061b959df40babbff24d17d151b19)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.009614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.243849 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d061b959df40babbff24d17d151b19', 'request_id': 'req-99a95389-7024-48c4-bb64-bbb6e409c1c9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:50.249807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d061b959df40babbff24d17d151b19', 'request_id': 'req-99a95389-7024-48c4-bb64-bbb6e409c1c9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:50.258462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d061b959df40babbff24d17d151b19', 'request_id': 'req-99a95389-7024-48c4-bb64-bbb6e409c1c9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:50.362205 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] The token cache is being invalidate because project d4d061b959df40babbff24d17d151b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:22:50.362746 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:50.363793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-99a95389-7024-48c4-bb64-bbb6e409c1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:22:50.365166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1418/2850] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:22:49 2022] DELETE /identity/v3/projects/d4d061b959df40babbff24d17d151b19 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:50.370400 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.370868 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.371126 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.393395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:50.394857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:50.414832 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:50.460579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1433/2851] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:50.467811 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.468162 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.468509 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.489741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:50.494425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:50.499062 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1419/2852] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:50.503465 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.513966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65cddf-8e22-4b4e-8599-541f0ced8827 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.520416 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.521049 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.521323 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.521596 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.522233 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:22:50.522551 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.524255 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.536858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1434/2853] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:22:50.541212 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.550255 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d40f23-ab4b-4d81-8902-75b48e72790e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.556281 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.556427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.556664 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.556973 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.557598 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.559529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.686584 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1420/2854] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:50 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:50.690952 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.701637 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c5c9-9b90-43ac-81e6-7cf4baf7ff35 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.756787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.757376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.757652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.757920 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.758528 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.760485 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.858814 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1435/2855] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:50 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:50.863404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.873844 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e173a-7dc8-4406-a446-a57a5cc5a784 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.880436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.880436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.880436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.880436 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.881003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.882585 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.892986 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1421/2856] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:50.897076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.907011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d023e-0425-4d6a-9e86-fc0aae978e0f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.911949 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.912535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.912810 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.913082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.913721 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.915415 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774837e3-76ac-4e04-819e-cd59dbffcc74 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:50.926437 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1436/2857] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:50.930409 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:50.940504 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febac9a5-cd48-460b-9641-9eea6bb28886 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:50.945976 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:50.946646 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:50.946928 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:50.947260 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] PATH_INFO: `/v3/projects/049e76e29a48415681a0047f72bb9e8c/users/702cf20256a74edcad54abbc464cb099/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:50.962321 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=049e76e29a48415681a0047f72bb9e8c, user_id=702cf20256a74edcad54abbc464cb099, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:50.964952 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:51.042284 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1422/2858] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:50 2022] PUT /identity/v3/projects/049e76e29a48415681a0047f72bb9e8c/users/702cf20256a74edcad54abbc464cb099/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:22:51.053475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9cda-8705-4758-aab9-4f751deeb56b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:51.053757 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9cda-8705-4758-aab9-4f751deeb56b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:51.054044 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d9cda-8705-4758-aab9-4f751deeb56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:51.073933 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-e34d9cda-8705-4758-aab9-4f751deeb56b None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:51.075253 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e34d9cda-8705-4758-aab9-4f751deeb56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:51.135741 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1437/2859] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:51.145836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:51.147546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:51.161456 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:51.205690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1534ce-be0a-464c-b2e7-92fcaf52dad5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:51.209697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:51.210295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:51.210574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:51.210845 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:51.219665 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:51.221121 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:51.230860 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1423/2860] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:22:56.924047 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:56.924573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:56.924573 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:56.954772 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] MFA Rules not processed for user `702cf20256a74edcad54abbc464cb099`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:22:56.956340 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:22:56.978436 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:22:57.030736 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1438/2861] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:22:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:22:57.036046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.049622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771efc-ccd5-4002-9d6c-7c874308d71a service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.056172 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.056825 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.057106 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.057376 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.058010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.060158 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.146198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1424/2862] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:22:57 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:57.150247 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.161795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f8fd3-06a1-4b29-9952-3fc29fed2361 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.220915 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.221562 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.222451 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.223090 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.223799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.225791 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.362900 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1439/2863] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:22:57 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:22:57.367677 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.378219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071790-a23e-41db-a536-ac93e18e1bd5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.383249 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.383850 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.384127 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.384397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.385029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.386623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.399353 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1425/2864] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 09:22:57.403373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.414143 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517ea0f-23f5-46c2-b876-d0bf639a8662 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.418726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.419405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.419683 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.419956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] PATH_INFO: `/v3/projects/1aa3709a086c410da2924eb4e75443e8/users/a773fc880dea48e79922db9818bc5c95/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.437210 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa3709a086c410da2924eb4e75443e8, user_id=a773fc880dea48e79922db9818bc5c95, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.441430 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.516308 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1440/2865] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:57 2022] PUT /identity/v3/projects/1aa3709a086c410da2924eb4e75443e8/users/a773fc880dea48e79922db9818bc5c95/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:22:57.521710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.536020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d423-5b8e-420a-81d9-64897cff43a4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.603421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.603421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.603421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.603421 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.603758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.603758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be2d7a-2457-46f7-824b-1124b9663a7a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.609561 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1426/2866] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:22:57.613980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:22:57.625556 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91f902-1851-438e-b72e-3564763ee9d3 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:22:57.631733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:22:57.634645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:22:57.634645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:22:57.634645 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] PATH_INFO: `/v3/projects/1aa3709a086c410da2924eb4e75443e8/users/a773fc880dea48e79922db9818bc5c95/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:22:57.635293 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa3709a086c410da2924eb4e75443e8, user_id=a773fc880dea48e79922db9818bc5c95, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:22:57.637961 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:22:57.649799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1441/2867] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:22:57 2022] PUT /identity/v3/projects/1aa3709a086c410da2924eb4e75443e8/users/a773fc880dea48e79922db9818bc5c95/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:23:02.994070 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f883009-80ea-4d2c-b333-2357021b5aa6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:23:02.994590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f883009-80ea-4d2c-b333-2357021b5aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:23:02.994711 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f883009-80ea-4d2c-b333-2357021b5aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:23:03.033812 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4f883009-80ea-4d2c-b333-2357021b5aa6 None None] MFA Rules not processed for user `a773fc880dea48e79922db9818bc5c95`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:23:03.036895 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f883009-80ea-4d2c-b333-2357021b5aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:03.133452 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1427/2868] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:23:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:23:03.142763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:23:03.145238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:03.236128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118ff3-3a77-47d8-850a-96d560dcef39 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:23:03.242579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:23:03.243536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:23:03.243536 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:23:03.243819 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:23:03.250048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:23:03.306983 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:23:03.308762 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:23:03.319610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1442/2869] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:23:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:23:05.966197 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:23:05.967235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:23:05.967235 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:23:05.973862 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:23:05.986499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] MFA Rules not processed for user `702cf20256a74edcad54abbc464cb099`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:23:05.988324 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:05.995836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1428/2870] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:23:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:23:13.348238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=24 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:23:16.094989 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:23:36.478048 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:23:36.479649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:36.553150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336e69-c9b1-4086-946b-9ddd4803dfa9 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:23:36.559168 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:23:36.559786 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:23:36.560091 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:23:36.560361 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:23:36.562098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:36.575012 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:23:36.576686 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a3f83-60f5-4c59-97c8-cbef18aed918 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:23:36.588132 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1443/2871] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:23:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:23:43.991166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:23:43.992671 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:44.048981 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1f6c3-7e4a-4af8-9707-bf15820c2a9a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:23:44.052483 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:23:44.053073 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:23:44.053348 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:23:44.053614 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:23:44.055458 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:23:44.065385 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:23:44.066933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:23:44.074894 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1429/2872] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:23:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:23:46.582422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=32 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:23:54.071272 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=26 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:24:05.640191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:24:05.640766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:24:05.640766 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:24:05.660521 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] MFA Rules not processed for user `7538ff30b3ff42a784da0336012f901d`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:24:05.661733 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:24:05.719045 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1444/2873] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:24:08.810336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:24:08.819325 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f818bd-e4b9-44ad-933b-ed445db80ffe service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:24:08.823055 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:24:08.823680 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:24:08.823958 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:24:08.824223 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:24:08.834275 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:24:08.835738 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:24:08.848438 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1430/2874] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:24:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:24:10.802153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:24:10.816192 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9799a-91cb-4ebb-84ff-b3c8290574f5 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:24:10.822659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:24:10.823583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:24:10.824076 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:24:10.824355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:24:10.826191 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:24:10.840441 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:24:10.842511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:24:10.854759 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1445/2875] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:24:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:24:18.843496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:24:20.850652 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=31 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:20.126248 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:20.129132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:20.143492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636d1e2-51ff-4d64-921d-995e50f36433 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:20.148601 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:20.149204 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:20.149485 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:20.149758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:20.151557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:20.164118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:20.165741 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:20.176076 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1431/2876] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:25:20.233377 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:20.234864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:20.301980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af94536-a84d-456b-8fdf-c94d078f1aaf service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:20.306534 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:20.307193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:20.307475 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:20.307744 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] PATH_INFO: `/v3/users/7538ff30b3ff42a784da0336012f901d` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:20.308849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7538ff30b3ff42a784da0336012f901d)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:20.310877 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:20.525202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7538ff30b3ff42a784da0336012f901d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:20.537501 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7538ff30b3ff42a784da0336012f901d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:20.544365 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7538ff30b3ff42a784da0336012f901d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:20.545058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7538ff30b3ff42a784da0336012f901d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:20.545673 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-94ba6fc2-cff1-45f3-aa00-829ca3187110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7538ff30b3ff42a784da0336012f901d'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:20.638114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1446/2877] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:25:20 2022] DELETE /identity/v3/users/7538ff30b3ff42a784da0336012f901d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:20.856492 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:20.868789 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3fa71-ea3b-444e-8d75-4bf0d2997854 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:20.924239 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:20.924853 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:20.925163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:20.925428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] PATH_INFO: `/v3/projects/8c703a441ccd4fd6aed56096783f1086` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:20.926630 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c703a441ccd4fd6aed56096783f1086)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:20.928526 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.015926 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c703a441ccd4fd6aed56096783f1086', 'request_id': 'req-6c5c87b1-7d05-428a-875e-647977eb7627'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:21.022462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c703a441ccd4fd6aed56096783f1086', 'request_id': 'req-6c5c87b1-7d05-428a-875e-647977eb7627'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:21.028656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c703a441ccd4fd6aed56096783f1086', 'request_id': 'req-6c5c87b1-7d05-428a-875e-647977eb7627'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:21.203572 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] The token cache is being invalidate because project 8c703a441ccd4fd6aed56096783f1086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:25:21.204296 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:21.204418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6c5c87b1-7d05-428a-875e-647977eb7627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:21.205911 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1432/2878] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:25:20 2022] DELETE /identity/v3/projects/8c703a441ccd4fd6aed56096783f1086 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:21.211464 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.211750 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.212021 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.231907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:21.233092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:21.362997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1447/2879] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:21.371734 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.372020 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.372300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.392558 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:21.394404 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:21.402603 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1433/2880] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:21.406735 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.416649 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039100db-0463-4d3e-a760-e5cecdf685f5 None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.421188 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.421782 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.422058 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.422327 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.422957 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:25:21.423314 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.424902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.436650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1448/2881] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 09:25:21.441542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.450557 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbba02-ad5a-424e-ab6a-874067205d2e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.455482 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.456010 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.456909 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.457189 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.457805 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.459733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.559720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1434/2882] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:25:21 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:25:21.564028 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.574954 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8faa2-e9bb-494f-8ce9-e37429502526 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.579002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:21.632041 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.632699 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.632974 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.633248 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.633901 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.635852 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.730000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1449/2883] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:25:21 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:25:21.734060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.744759 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ea623-1690-47c1-9bcf-f6352c610bb3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.749473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.750061 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.750340 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.750641 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.751308 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.752905 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.763394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1435/2884] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:21.770895 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.780730 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53efaa2-f190-454d-ba3f-fc54d4e66d3c admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.785955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.786543 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.786824 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.787096 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.787836 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.789423 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47ac0f-d8d4-4dcd-a652-f51741726a78 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.802052 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1450/2885] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:21.806154 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:21.816818 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f9a31-9835-4345-a087-58381885b19a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:21.821400 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:21.822032 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.822309 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.822579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] PATH_INFO: `/v3/projects/cee360a215044ccd819d9cd0000d7dc8/users/86571f1113f34bc4bf877df5e7a3a447/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.836982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee360a215044ccd819d9cd0000d7dc8, user_id=86571f1113f34bc4bf877df5e7a3a447, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:21.839656 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:21.917499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1436/2886] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:25:21 2022] PUT /identity/v3/projects/cee360a215044ccd819d9cd0000d7dc8/users/86571f1113f34bc4bf877df5e7a3a447/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:21.928092 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:21.928373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:21.928778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:21.957356 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] MFA Rules not processed for user `86571f1113f34bc4bf877df5e7a3a447`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:21.958670 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:22.021945 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1451/2887] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:22.032542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:22.034266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:22.048131 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:22.091071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e59ff-8817-4ee6-9bff-ed3c52cafdbd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:22.094756 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:22.095389 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:22.095740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:22.095961 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:22.104301 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:22.105807 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012559b-9d71-4d76-83df-7b6cd1f64857 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:22.115336 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1437/2888] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:25:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:25:32.018320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:32.129580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=20 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:45.907851 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:45.969643 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fd652-c81b-4391-9e5a-626652915d9b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:45.974081 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:45.974904 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:45.975172 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:45.975284 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] PATH_INFO: `/v3/users/86571f1113f34bc4bf877df5e7a3a447` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:45.976328 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86571f1113f34bc4bf877df5e7a3a447)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:45.978453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:46.170555 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86571f1113f34bc4bf877df5e7a3a447'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.182910 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86571f1113f34bc4bf877df5e7a3a447'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.189752 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86571f1113f34bc4bf877df5e7a3a447'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.190395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86571f1113f34bc4bf877df5e7a3a447'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.190997 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86571f1113f34bc4bf877df5e7a3a447'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.265392 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1452/2889] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:25:45 2022] DELETE /identity/v3/users/86571f1113f34bc4bf877df5e7a3a447 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:46.270004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:46.270472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:46.270575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:46.289620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:46.290881 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:46.348411 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1438/2890] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:46.358109 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:46.359580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:46.416690 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178a4f2-9ca2-4c10-9a68-40494ffc7017 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:46.420523 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:46.421173 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:46.421398 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:46.421666 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:46.431275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:46.432697 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33875e03-4984-496f-b9c4-134654c0396d service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:46.441943 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1453/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:25:46.822113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:46.823675 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:46.837139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2e5e5-b9e9-45ff-acb0-af4d4c386cc4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:46.842019 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:46.843125 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:46.843408 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:46.843678 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] PATH_INFO: `/v3/projects/cee360a215044ccd819d9cd0000d7dc8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:46.846852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee360a215044ccd819d9cd0000d7dc8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:46.849029 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:46.955428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee360a215044ccd819d9cd0000d7dc8', 'request_id': 'req-912cfc74-514d-44fe-8f96-7529141df7b6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.961635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee360a215044ccd819d9cd0000d7dc8', 'request_id': 'req-912cfc74-514d-44fe-8f96-7529141df7b6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:46.968007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee360a215044ccd819d9cd0000d7dc8', 'request_id': 'req-912cfc74-514d-44fe-8f96-7529141df7b6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:47.057496 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] The token cache is being invalidate because project cee360a215044ccd819d9cd0000d7dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:25:47.057781 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:47.058397 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-912cfc74-514d-44fe-8f96-7529141df7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:25:47.059935 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1439/2892] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:25:46 2022] DELETE /identity/v3/projects/cee360a215044ccd819d9cd0000d7dc8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 09:25:47.064862 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.065341 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.065447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.085532 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:47.087843 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:47.105504 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:47.147206 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1454/2893] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:47.156994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.157306 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.157580 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.177631 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:47.178819 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:47.186567 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1440/2894] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:47.190648 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.200229 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b440-3c4f-45e2-8017-d6f66a182faf None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.205019 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.205659 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.205883 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.206153 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] PATH_INFO: `/v3/domains` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.206787 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81633) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 09:25:47.207084 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.208703 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.220768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1455/2895] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 09:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 09:25:47.230542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.239879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544ea6a-64a5-4a46-8f21-baf8a2014931 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.244674 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.245261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.245539 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.245809 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.246428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.248412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.335541 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1441/2896] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:25:47 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 09:25:47.340319 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.350309 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8884d-8dc8-4f79-86f1-4042f76c112e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.406799 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.407446 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.407729 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.408004 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.408624 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.410537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.510412 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1456/2897] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:25:47 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:25:47.514617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.526206 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e897c64-a8e6-4156-8a74-b18f98d72f76 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.530794 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.531520 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.536434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.536434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.536434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.536434 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.545085 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1442/2898] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:47.549200 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.559632 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0aba-c369-436c-8176-8c6be46ef43e admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.564301 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.565142 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.565307 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.565467 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.566098 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.567795 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246232d4-b83d-4b6d-a42b-0cca9695ce50 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.579373 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1457/2899] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:47.583579 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.594497 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd34cb-0b83-44f7-b62c-c81d099d2d39 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.599005 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.599713 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.599994 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.600267 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] PATH_INFO: `/v3/projects/94894dd3e80e424b8f34affb48a574ce/users/440380722a284fbe97aad70139e5ff1f/roles/5fecc05bf35b4b51af54c339e7a42bb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.616079 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94894dd3e80e424b8f34affb48a574ce, user_id=440380722a284fbe97aad70139e5ff1f, role_id=5fecc05bf35b4b51af54c339e7a42bb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.619533 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.686755 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1443/2900] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:25:47 2022] PUT /identity/v3/projects/94894dd3e80e424b8f34affb48a574ce/users/440380722a284fbe97aad70139e5ff1f/roles/5fecc05bf35b4b51af54c339e7a42bb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:47.692078 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26150d6-eede-4b55-92a7-d24a6633ede4 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.692364 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26150d6-eede-4b55-92a7-d24a6633ede4 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.692639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26150d6-eede-4b55-92a7-d24a6633ede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.712472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-b26150d6-eede-4b55-92a7-d24a6633ede4 None None] MFA Rules not processed for user `78b783de521c4bc3ade218e12aef00c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:47.719907 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-b26150d6-eede-4b55-92a7-d24a6633ede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:47.771765 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1458/2901] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:47.782276 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:47.784003 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:47.806166 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:47.839892 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f52d-8ed5-43e7-920a-5dbc8428ab24 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:47.843904 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:47.844519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:47.844793 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:47.845060 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:47.855467 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:47.856879 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:47.866001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1444/2902] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:25:53.351355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.351355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.351355 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.370590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] MFA Rules not processed for user `440380722a284fbe97aad70139e5ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:53.371865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:53.388842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:53.436322 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1459/2903] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:53.440966 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.451014 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabde98-9a7f-4025-a2a0-91df6937918b service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.455982 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.456573 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.456852 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.457119 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.457743 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.459705 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.548280 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1445/2904] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 09:25:53 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:25:53.552511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.566330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18449c-749b-48b0-9a0f-5771e7d660ed None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.622539 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.623067 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.623393 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.623669 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] PATH_INFO: `/v3/users` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.624344 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.626275 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.694277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1460/2905] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 09:25:53 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 09:25:53.698546 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.709681 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7209-b0b1-45a9-8a72-9db9ff9a5a50 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.715132 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.715745 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.716023 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.716295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.716933 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.718542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.730676 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1446/2906] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:53.735027 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.746107 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d4158-5cf4-456f-b5d7-238fa136c51d admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.750972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.751658 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.751936 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.752209 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] PATH_INFO: `/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837/users/272394a7f3d44041ac14e5684f397e29/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.769330 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171b6b13fcc54fa4b9ac0aa47da2a837, user_id=272394a7f3d44041ac14e5684f397e29, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.772064 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.843955 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1461/2907] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:25:53 2022] PUT /identity/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837/users/272394a7f3d44041ac14e5684f397e29/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:53.848004 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.859038 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e3174-9578-4818-bcdb-9693aab17c5d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.916536 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.917198 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.917501 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.917919 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.918566 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.920199 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080a664-e5e3-4ba5-9594-90f5a6794c59 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.930673 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1447/2908] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 09:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 09:25:53.934934 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:53.945584 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea613738-f8ec-4a6f-81b4-fc9c18d38252 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:53.950177 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:53.950816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] REQUEST_METHOD: `PUT` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:53.951095 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:53.951391 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] PATH_INFO: `/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837/users/272394a7f3d44041ac14e5684f397e29/roles/8821f299abc1414b9817dedf7f7d0bea` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:53.953583 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] RBAC: Authorizing `identity:create_grant(project_id=171b6b13fcc54fa4b9ac0aa47da2a837, user_id=272394a7f3d44041ac14e5684f397e29, role_id=8821f299abc1414b9817dedf7f7d0bea)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:53.956265 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:53.966956 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1462/2909] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 09:25:53 2022] PUT /identity/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837/users/272394a7f3d44041ac14e5684f397e29/roles/8821f299abc1414b9817dedf7f7d0bea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:25:59.544374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db30f0-cd7a-40c1-a8a7-fa2d17461ad5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:59.545021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db30f0-cd7a-40c1-a8a7-fa2d17461ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:59.545021 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db30f0-cd7a-40c1-a8a7-fa2d17461ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:59.579861 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-16db30f0-cd7a-40c1-a8a7-fa2d17461ad5 None None] MFA Rules not processed for user `272394a7f3d44041ac14e5684f397e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:25:59.581469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-16db30f0-cd7a-40c1-a8a7-fa2d17461ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:59.674867 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1448/2910] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:25:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:25:59.696564 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:25:59.699013 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:25:59.779042 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaef490-30d0-4bfa-be67-36d8faa8742b admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:25:59.785286 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:25:59.785829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:25:59.786105 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:25:59.786376 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:25:59.793077 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:25:59.842535 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:25:59.844150 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:25:59.853627 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1463/2911] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:25:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:26:09.670356 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:26:09.888517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=24 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:26:24.351545 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:24.352093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:24.352093 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:24.371543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] MFA Rules not processed for user `440380722a284fbe97aad70139e5ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:26:24.372786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:24.379633 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1449/2912] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:26:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:26:24.425785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:26:24.431299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:24.483258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e55465-c984-4c99-b72d-03bf6e355db9 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:26:24.486825 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:26:24.487456 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:24.488622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:24.488622 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:24.489760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:24.503405 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:26:24.504760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:26:24.515948 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1464/2913] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:26:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:26:25.034657 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:26:25.036071 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:25.091371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0bcc-3bb4-4b6b-938c-2dde821b1790 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:26:25.094851 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:26:25.095484 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:25.095764 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:25.096031 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:25.097710 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:25.106923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:26:25.108367 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:26:25.116175 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1450/2914] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:26:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:26:25.833511 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:26:25.843036 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34971c29-8961-49da-9a6a-042f7fdcb554 service cinder] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:26:25.846812 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:26:25.847433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:25.847711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:25.847981 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:25.849709 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:25.863345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:26:25.864790 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:26:25.877472 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1465/2915] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:26:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:26:35.112524 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=29 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:26:35.872888 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=32 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:26:43.692498 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:26:43.701265 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188afa4-baa7-4122-99cf-9a5b101f2ffd service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:26:43.705612 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:26:43.706186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:43.706461 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:43.706728 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:43.715361 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:26:43.716778 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcb0fe-e0b5-4f28-b1b6-314ecf499b19 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:26:43.726115 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1451/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:26:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:26:45.550128 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:26:45.551829 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:45.566779 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095b9bd-bbb9-491c-a8ac-64f54fdb3f89 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:26:45.572847 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:26:45.573527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:26:45.573857 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:26:45.574193 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:26:45.576280 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:26:45.589661 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:26:45.591334 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be53c39-9aaa-4771-807e-c48f351a430f service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:26:45.603369 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1466/2917] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:26:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 09:26:53.722103 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:26:55.598696 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=6 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:27:10.182139 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc356-48eb-4bcb-aff5-7d986015cad1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.182708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc356-48eb-4bcb-aff5-7d986015cad1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.182708 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfc356-48eb-4bcb-aff5-7d986015cad1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.184165 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1452/2918] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 09:27:10.191320 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e125-2d54-4d5e-88b6-43482270b6a5 None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.191873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e125-2d54-4d5e-88b6-43482270b6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.191873 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e125-2d54-4d5e-88b6-43482270b6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.235590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d215e125-2d54-4d5e-88b6-43482270b6a5 None None] MFA Rules not processed for user `d40b4a33e1e845c4b5faaf0b1e608c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:27:10.236816 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d215e125-2d54-4d5e-88b6-43482270b6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:27:10.247778 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1467/2919] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:27:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 09:27:10.261423 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfc356-48eb-4bcb-aff5-7d986015cad1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:27:10.273087 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfc356-48eb-4bcb-aff5-7d986015cad1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:27:10.276563 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:27:10.277140 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.277418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.277690 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.285622 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:27:10.287034 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a940ed-ac08-497e-b12a-63cda2e5eb11 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:27:10.294757 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1453/2920] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:27:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:27:10.380011 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9715e6-eb83-4263-aa95-3cca2fe88c8a None None] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.380297 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9715e6-eb83-4263-aa95-3cca2fe88c8a None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.380566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9715e6-eb83-4263-aa95-3cca2fe88c8a None None] PATH_INFO: `/` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.381941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1468/2921] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 09:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 09:27:10.386687 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.386975 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.387279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.424758 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] MFA Rules not processed for user `d132894fe36047418e5cd8bebe04c387`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:27:10.425964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:27:10.435134 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1454/2922] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 09:27:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:27:10.445638 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9715e6-eb83-4263-aa95-3cca2fe88c8a None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:27:10.457055 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9715e6-eb83-4263-aa95-3cca2fe88c8a None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:27:10.460590 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:27:10.461174 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:27:10.461453 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:27:10.461726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:27:10.472566 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:27:10.473991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:27:10.489352 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1469/2923] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:27:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:27:20.421917 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=15 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:27:20.480026 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=15 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:28:31.154668 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:28:31.174571 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c249f2d-062f-4aac-bb38-5c0ae7b13e94 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:28:31.180195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:28:31.180195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:28:31.180195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:28:31.180195 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:28:31.181113 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:28:31.210394 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:28:31.211838 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438bfe8-b4f6-4315-bdb7-83bb1186d6eb service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:28:31.220502 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1455/2924] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:28:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:28:37.431707 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:28:37.433103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:28:37.555506 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3511291-4c4d-4f35-b005-ed231b88831b service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:28:37.561230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:28:37.561871 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:28:37.562159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:28:37.562433 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:28:37.564238 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:28:37.576597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:28:37.578387 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:28:37.588425 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1470/2925] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:28:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:28:41.216617 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=19 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:28:47.584083 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=55 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:29:04.460374 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:04.460880 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:04.460985 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:04.479317 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] MFA Rules not processed for user `440380722a284fbe97aad70139e5ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:29:04.480500 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:04.487261 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1456/2926] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:29:05.475991 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:05.486657 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbdd24-23c6-40f6-9b32-637d7753db44 service placement] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:05.492002 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:05.492751 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:05.493031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:05.493305 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:05.495215 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:05.507897 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:05.508986 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:05.520159 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1471/2927] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:29:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:29:10.135233 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:10.156761 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ece37-c237-4611-98ad-2f62efed36b6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:10.160877 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:10.161068 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:10.161335 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:10.161466 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:10.163620 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:10.227028 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:10.228503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:10.238180 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1457/2928] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:29:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:29:13.711481 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:13.731388 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d186407-ef8a-4e01-8f13-84f76e310d0a service placement] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:13.737050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:13.737641 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:13.737921 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:13.738195 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:13.739953 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:13.752040 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:13.753625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:13.764756 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1472/2929] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:29:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:29:16.486112 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:16.494836 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee154f5e-e51d-45a3-8924-32bc7430195f service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:16.498722 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:16.499344 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:16.499623 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:16.500595 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:16.502635 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:16.590008 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:16.591486 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:16.610638 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1458/2930] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:29:22.981653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:22.993842 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0016-f0d5-45a9-ba2b-3254c3c887b2 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:23.000580 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:23.001202 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:23.001656 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:23.001941 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:23.005133 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:23.019964 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:23.021544 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:23.035440 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1473/2931] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:29:24.102302 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:24.103786 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:24.116234 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735a363-326c-4697-83f7-975a2ee06c70 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:24.121796 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:24.122428 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:24.122784 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:24.123371 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] PATH_INFO: `/v3/users/702cf20256a74edcad54abbc464cb099` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:24.124516 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702cf20256a74edcad54abbc464cb099)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:24.126472 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:24.300469 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:29:24.302768 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702cf20256a74edcad54abbc464cb099'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.314266 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702cf20256a74edcad54abbc464cb099'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.392615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702cf20256a74edcad54abbc464cb099'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.393174 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702cf20256a74edcad54abbc464cb099'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.393772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702cf20256a74edcad54abbc464cb099'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.407473 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1459/2932] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:29:24 2022] DELETE /identity/v3/users/702cf20256a74edcad54abbc464cb099 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:29:24.663395 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:24.675517 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70a49-7351-4e5f-96cc-95d62c6e00b7 service nova] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:24.733144 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:24.733760 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:24.734674 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:24.735031 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] PATH_INFO: `/v3/projects/049e76e29a48415681a0047f72bb9e8c` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:24.736180 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=049e76e29a48415681a0047f72bb9e8c)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:24.738114 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:24.839420 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049e76e29a48415681a0047f72bb9e8c', 'request_id': 'req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.895237 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049e76e29a48415681a0047f72bb9e8c', 'request_id': 'req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.901620 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049e76e29a48415681a0047f72bb9e8c', 'request_id': 'req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.912864 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] The token cache is being invalidate because project 049e76e29a48415681a0047f72bb9e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:29:24.913086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.913706 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:24.915212 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1474/2933] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:29:24 2022] DELETE /identity/v3/projects/049e76e29a48415681a0047f72bb9e8c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:29:24.919196 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:24.920574 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:24.934820 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c4be-ee93-465a-93ca-a2599da3b541 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:24.992513 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:24.993141 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:24.993418 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:24.993686 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] PATH_INFO: `/v3/users/a773fc880dea48e79922db9818bc5c95` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:24.994806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a773fc880dea48e79922db9818bc5c95)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:24.996878 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:25.178964 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a773fc880dea48e79922db9818bc5c95'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.191814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a773fc880dea48e79922db9818bc5c95'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.250523 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a773fc880dea48e79922db9818bc5c95'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.251215 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a773fc880dea48e79922db9818bc5c95'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.251822 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a773fc880dea48e79922db9818bc5c95'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.264733 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1460/2934] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:29:24 2022] DELETE /identity/v3/users/a773fc880dea48e79922db9818bc5c95 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:29:25.497460 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:29:25.498899 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:29:25.513277 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e95ef9-4c88-44e3-8f57-8a8ae0c8c99f admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:29:25.558085 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:29:25.580488 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:29:25.581151 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:29:25.581426 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:29:25.581692 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] PATH_INFO: `/v3/projects/1aa3709a086c410da2924eb4e75443e8` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:29:25.583285 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa3709a086c410da2924eb4e75443e8)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:29:25.585258 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:29:25.695335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa3709a086c410da2924eb4e75443e8', 'request_id': 'req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.774610 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa3709a086c410da2924eb4e75443e8', 'request_id': 'req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.782644 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa3709a086c410da2924eb4e75443e8', 'request_id': 'req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.793033 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] The token cache is being invalidate because project 1aa3709a086c410da2924eb4e75443e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:29:25.793337 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.793992 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-ce15b9d0-f6fc-42aa-a046-4c47c7e3d35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:29:25.795597 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1475/2935] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:29:25 2022] DELETE /identity/v3/projects/1aa3709a086c410da2924eb4e75443e8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:29:35.261422 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:29:35.878844 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:31:30.551776 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:31:30.553330 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:30.632358 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d38084-3a06-4178-b5e4-5f9fea265d73 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:31:30.680090 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:31:30.680704 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:31:30.681575 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:31:30.681787 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:31:30.683720 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:30.740923 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:31:30.742355 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:31:30.751782 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1461/2936] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:31:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:31:40.747775 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=77 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:31:42.441884 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] REQUEST_METHOD: `POST` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:31:42.442422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:31:42.442422 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:31:42.461796 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.auth.core [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] MFA Rules not processed for user `440380722a284fbe97aad70139e5ff1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81633) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 09:31:42.462975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:42.471069 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1476/2937] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 09:31:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 09:31:42.514899 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:31:42.516341 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:42.528006 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb584b-2eeb-4d7f-9b16-a506eb05b72a service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:31:42.531891 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33f0625dda614821a5d9ca6c75de33d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:31:42.532491 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:31:42.532771 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:31:42.533046 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:31:42.534750 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:42.619118 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:31:42.621300 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:31:42.642051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1462/2938] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:31:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:31:43.159298 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:31:43.160763 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:43.228527 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e37c6-9e27-41ca-988f-bcb5f57afd6b None None] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:31:43.232711 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:31:43.233302 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:31:43.233579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:31:43.233849 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:31:43.235579 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:43.245495 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:31:43.246980 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:31:43.254780 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1477/2939] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:31:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:31:43.927772 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:31:43.929305 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:43.940983 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691316-bbda-4474-a7da-d2711df25575 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:31:43.944863 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:31:43.945736 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:31:43.946016 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:31:43.946283 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:31:43.948538 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:31:43.963412 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:31:43.965403 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:31:43.976959 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1463/2940] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:31:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:31:53.251250 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=34 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:31:53.971865 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=49 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:32:02.947431 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:32:02.956568 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a51c68-16fe-4032-b346-7dd57f17545e service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:32:02.960086 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:32:02.960668 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:32:02.960966 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:32:02.961234 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:32:02.962911 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:32:03.027865 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:32:03.029385 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:32:03.039060 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1478/2941] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:32:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:32:04.841932 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:32:04.843462 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:32:04.857425 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ebb4e-2ccc-492d-8460-9048d7db677c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:32:04.863056 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd132894fe36047418e5cd8bebe04c387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:32:04.863684 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:32:04.863965 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:32:04.864238 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:32:04.865936 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:32:04.878529 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:32:04.880395 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:32:04.891634 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1464/2942] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:32:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:32:13.034655 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=30 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:32:14.887123 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:33:35.130103 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:33:35.138803 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f5e65-53bd-4275-8cf2-0245d421f29d service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:33:35.142498 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:33:35.143082 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:33:35.143383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:33:35.143650 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:33:35.145345 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:33:35.154609 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:33:35.156050 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:33:35.163972 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1479/2943] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:33:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:33:42.440814 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:33:42.442279 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:33:42.509405 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a58b4e-b4f3-44b4-8e95-88fe069cedbe service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:33:42.514543 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:33:42.515163 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:33:42.515450 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:33:42.515724 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:33:42.517424 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:33:42.528530 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:33:42.530277 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:33:42.540216 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1465/2944] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:33:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:33:45.160447 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=6 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:33:52.535765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=32 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:34:07.950693 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:07.960074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ec7f4-ca2b-489a-ae73-a87cbacda719 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:07.964016 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd40b4a33e1e845c4b5faaf0b1e608c59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:07.964639 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:07.964914 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:07.965184 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:07.966913 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:08.030574 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:08.032112 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:08.041726 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1480/2945] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:34:11.435089 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:11.436593 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:11.450806 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7981b-246a-4496-abe4-4f565b291452 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:11.457078 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:11.457799 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:11.458082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:11.458354 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:11.460169 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:11.472398 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:11.474001 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:11.484314 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1466/2946] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:34:14.743335 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:14.755783 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bd74-ac9f-4743-a112-6f48429f26d4 service neutron] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:14.760937 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:14.761768 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] REQUEST_METHOD: `GET` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:14.762043 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:14.762316 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:14.764074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:14.776157 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:14.777725 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:14.788321 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1481/2947] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:34:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:34:14.888740 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:14.890246 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:14.904493 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e5aa4-01b6-4435-ae84-e316de667b13 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:14.910082 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:14.910697 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:14.910972 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:14.911278 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] PATH_INFO: `/v3/users/440380722a284fbe97aad70139e5ff1f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:14.912441 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] RBAC: Authorizing `identity:delete_user(user_id=440380722a284fbe97aad70139e5ff1f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:14.914295 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:15.101167 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440380722a284fbe97aad70139e5ff1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.113887 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440380722a284fbe97aad70139e5ff1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.197173 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440380722a284fbe97aad70139e5ff1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.197715 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440380722a284fbe97aad70139e5ff1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.198323 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440380722a284fbe97aad70139e5ff1f'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.211052 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1467/2948] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:34:14 2022] DELETE /identity/v3/users/440380722a284fbe97aad70139e5ff1f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:34:15.443727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:15.454975 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74873397-8085-4b09-b195-a474cd4b81e8 service placement] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:15.511892 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:15.512520 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:15.512794 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:15.513080 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] PATH_INFO: `/v3/projects/94894dd3e80e424b8f34affb48a574ce` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:15.514156 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94894dd3e80e424b8f34affb48a574ce)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:15.516714 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:15.631124 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94894dd3e80e424b8f34affb48a574ce', 'request_id': 'req-e63147c3-32b7-483d-9472-53b1a46d1825'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.694383 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94894dd3e80e424b8f34affb48a574ce', 'request_id': 'req-e63147c3-32b7-483d-9472-53b1a46d1825'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.703503 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94894dd3e80e424b8f34affb48a574ce', 'request_id': 'req-e63147c3-32b7-483d-9472-53b1a46d1825'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.712625 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] The token cache is being invalidate because project 94894dd3e80e424b8f34affb48a574ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:34:15.712917 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.713593 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:15.715230 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1482/2949] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:34:15 2022] DELETE /identity/v3/projects/94894dd3e80e424b8f34affb48a574ce => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:34:15.719331 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:15.720780 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:15.733615 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2e467-16c7-4148-aa7d-41ee256f639c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:15.791274 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:15.791903 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:15.792186 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:15.792463 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] PATH_INFO: `/v3/users/272394a7f3d44041ac14e5684f397e29` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:15.793590 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=272394a7f3d44041ac14e5684f397e29)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:15.795542 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:16.005251 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272394a7f3d44041ac14e5684f397e29'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.015051 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:34:16.017427 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272394a7f3d44041ac14e5684f397e29'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.088519 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272394a7f3d44041ac14e5684f397e29'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.089135 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272394a7f3d44041ac14e5684f397e29'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.089779 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272394a7f3d44041ac14e5684f397e29'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.103007 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1468/2950] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 09:34:15 2022] DELETE /identity/v3/users/272394a7f3d44041ac14e5684f397e29 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:34:16.382000 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Authenticating user token {{(pid=81633) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:34:16.384496 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.fernet_utils [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81633) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:34:16.396727 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63147c3-32b7-483d-9472-53b1a46d1825 admin admin] Validating token access rules against request {{(pid=81633) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:34:16.405074 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:34:16.456537 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b783de521c4bc3ade218e12aef00c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0c919d0d1fdc41adb58cac59aa4513ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81633) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:34:16.457186 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:34:16.457538 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:34:16.457732 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] PATH_INFO: `/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837` {{(pid=81633) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:34:16.458858 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171b6b13fcc54fa4b9ac0aa47da2a837)` {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:34:16.460854 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] RBAC: Authorization granted {{(pid=81633) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:34:16.558324 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171b6b13fcc54fa4b9ac0aa47da2a837', 'request_id': 'req-90f3d5ba-d58d-46f3-a898-974b28c6c788'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.639474 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171b6b13fcc54fa4b9ac0aa47da2a837', 'request_id': 'req-90f3d5ba-d58d-46f3-a898-974b28c6c788'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.642785 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171b6b13fcc54fa4b9ac0aa47da2a837', 'request_id': 'req-90f3d5ba-d58d-46f3-a898-974b28c6c788'} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.653357 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] The token cache is being invalidate because project 171b6b13fcc54fa4b9ac0aa47da2a837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81633) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 09:34:16.653653 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.654299 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG keystone.notifications [None req-90f3d5ba-d58d-46f3-a898-974b28c6c788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81633) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 09:34:16.655902 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: [pid: 81633|app: 0|req: 1483/2951] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 09:34:16 2022] DELETE /identity/v3/projects/171b6b13fcc54fa4b9ac0aa47da2a837 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 09:34:26.139817 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:34:26.650211 dv-cinder-820027-23-iscsi devstack@keystone.service[81633]: DEBUG dbcounter [-] [81633] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 09:38:45.397256 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 09:38:45.398689 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:38:45.532586 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4280061-e9b9-40fb-ab58-f325d0ab1e1a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 09:38:45.590578 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97133f7b0f5b4320a41486752de09440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c638c2670f324f9095be90304ed677a9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 09:38:45.591219 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 09:38:45.591499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 09:38:45.591765 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 09:38:45.593475 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 09:38:45.605477 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 09:38:45.607039 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828efebf-aead-496e-a961-e26fc31cebc6 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 09:38:45.617499 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1469/2952] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 09:38:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 09:38:55.612503 dv-cinder-820027-23-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}